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FOREWORD 


This  report  is  a  comprehensive  examination  of  the  research  evidence  re¬ 
garding  computer-based  instruction  (CBI)  on  costs,  learning  achievements, 
individual  differences,  and  student  attitudes.  Even  though  CBI  promises  to 
have  a  significant  impact  upon  students'  learning,  inconsistencies  and  prob¬ 
lems  with  the  current  research  evidence  prohibit  any  conclusions  about  the 
claimed  advantages  of  CBI  over  other  instructional  media.  Recommendations 
are  made  about  the  methodological  considerations  involved  in  evaluating  a  CBI 
system  and  the  usefulness  of  this  instructional  technology.  A  salient  recom¬ 
mendation  from  this  technical  report  is  that  researchers  should  shift  focus 
from  examining  the  inherent  superiority  of  this  medium  to  identifying  the 
conditions  for  using  computers  in  the  instructional  process.  The  conclusions 
are  intended  for  military  personnel  and  behavioral  scientists  interested  in 
implementing  or  investigating  a  CBI  system. 


EDGAR  M.  JOHNSON 
Technical  Director 


AN  EXAMINATION  OF  THE  RESEARCH  EVIDENCE  FOR  COMPUTER-BASED  INSTRUCTION 
IN  MILITARY  TRAINING 


EXECUTIVE  SUMMARY 


Requirement : 

This  discussion  will  help  the  military  training  community  understand  the 
research  literature  on  the  usefulness  of  CBI  as  a  primary  instructional  medium. 
The  issues  presented  in  this  paper  will  also  assist  behavioral  scientists  in 
investigating  and  understanding  the  strengths  and  weaknesses  of  this  educa¬ 
tional  medium. 


Procedure : 

Over  150  papers  and  articles  were  examined.  Interviews  with  noted  CBI 
professionals  were  also  conducted  to  obtain  information  about  CBI  and  the  re¬ 
search  evidence.  This  discussion  focuses  upon  four  main  areas  of  CBI  research — 
costs,  learning  achievements,  individual  differences,  and  student  attitudes. 
Implications  regarding  future  CBI  use  and  research  were  also  discussed. 


Findings : 

Consistent  empirical  evidence  does  not  exist  to  support  or  deny  claimed 
advantages  of  CBI  over  other  instructional  media  for  (a)  reducing  training 
time;  (b)  reducing  life-cycle  costs;  (c)  facilitating  students'  mastery  of  the 
instructional  materials;  (d)  accommodating  individual  learning  differences;  and 
(e)  motivating  students'  learning.  The  lack  of  empirical  support  for  these  is¬ 
sues  is  not  totally  explained  by  problematic  courseware.  CBI's  inherent  supe¬ 
riority  as  a  primary  instructional  medium  has  thus  not  been  established. 

Future  generations  of  CBI  do  promise  to  have  a  significantly  positive 
impact  upon  students'  cognitive  process.  This  medium  also  promises  to  help 
slow  learners.  And  it  does  not  seem  that  CBI  is  a  dehumanizing  instructional 
experience. 

Problematic  research  procedures  were  also  found  throughout  the  CBI  litera¬ 
ture.  Most  noticeable  of  these  research  problems  were  (a)  confoundings  due  to 
differences  in  instructional  content;  (b)  making  comparisons  with  inappropriate 
media;  (c)  confoundings  due  to  "program  novelty  effects";  and  (d)  findings  that 
were  not  replicated. 


Utilization  of  Findings: 


Widespread  implementation  of  CBI  for  all  educational  situations  is  not 
recommended.  Such  indiscriminate  use  of  CBI  could  lead  to  the  abandonment  of 
this  instructional  technology,  because  the  research  evidence  has  indicated  that 
at  present  the  promises  may  outweigh  the  potential.  CBI  might  be  useful  as  an 
instructional  tool  to  make  the  established  instructional  program  more  effective. 
CBI  researchers  must  help  determine  those  situations  for  which  instructors  can 
best  use  this  medium.  And  the  methodological  recommendations  discussed  in  this 
report  should  be  incorporated  in  future  CBI  evaluations. 
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AN  EXAMINATION  OF  THE  RESEARCH  EVIDENCE  FOR  COMPUTER-BASED 
INSTRUCTION  IN  MILITARY  TRAINING* 


Introduction 

Computer-Based  Instruction  (CBI)  proponents  (e.g.,  Kearsley,  Hunter,  & 
Seidel,  1983;  Myers,  1984;  Shavelson  et  al.,  1984  and  Wilson,  1984)  have 
suggested  that  this  technology's  delivery  capabilities  make  CBI  superior  to 
other  instructional  modes.  As  Shavelson  et  al .  stated: 

Federal  and  state  policy  makers  all  contend  that  recent  techno¬ 
logical  innovations  most  notably  the  microcomputers  hold  par¬ 
ticular  promise  not  only  for  pulling  education  out  of  a  rising 
tide  of  mediocrity,  but  also  for  reshaping  education,  (pi) 

CBI  denotes  using  computers  in  the  instructional  process.  Computer- 
Assisted  Instruction  (CAI)  and  Computer-Managed  Instruction  (CMI)  are  consid¬ 
ered  to  be  components  of  CBI.  In  CAI,  students  receive  instruction  directly 
from  the  computer  system.  CMI  involves  using  the  computer  for  such  instruc¬ 
tional  management  issues  as  scoring,  recording,  and  interpreting  test 
results.  Instructions  for  a  CMI  system,  however,  are  provided  off-line 
either  by  the  teacher  or  by  other  instructional  media.  Other  uses  of  CMI 
will  be  discussed  in  succeeding  sections. 

CBI  is  especially  being  touted  as  a  significant  instructional  device  for 
the  United  States'  Armed  Services  (see  Department  of  Defense  [DoD])  Memoran¬ 
dum  from  the  Assistant  Secretary  of  Defense,  9  September  1985).  Orlansky 
(1985)  has  noted  that  CBI  is  particularly  applicable  in  training  military 
personnel  for  a  significant  number  of  the  over  10,000  Armed  Services'  skill 
training  courses  offered  in  such  areas  as  electronics,  maintenance,  and  com¬ 
munications.  These  courses  will  train  approximately  1.4  million  students  at 
a  cost  of  nearly  4.4  billion  dollars  in  fiscal  year  1 986-  The  Army  is  plan¬ 
ning  to  develop  CBI  programs  for  training  different  levels  of  troops  in  many 
different  types  of  courses,  including  advanced  courses  for  officers.  Che 
example  of  the  Army's  plans  to  use  CBI  is  the  approximately  4 20  hours  of  the 
Job  Skills  Education  Program  (JSEP),  which  will  be  put  on  computers  (Farr, 
1985).  The  JSEP  package  will  involve  the  use  of  200-300  terminals  located  at 
19  different  military  sites.  This  program  will  provide  remedial  training  for 
soldiers  with  problems  mastering  the  skills  needed  in  their  Military  Occupa¬ 
tion  Specialty  (MOS).  The  Army's  expanded  plans  to  use  CBI  also  involve  the 
US  Army's  Armor  School  at  Fort  Knox.  The  Armor  School  is  in  the  process  of 

•Portions  of  this  report  will  appear  as  a  chapter  in  R.T.  Hartson  and  D.  Hix 
(Eds),  Advances  in  Human-Computer  Interactions.  Ablex  Publishing  Corpora¬ 
tion:  Norwood,  New  Jersey. 
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developing  over  350  hours  of  courseware,  which  represent  approximately  20%  of 
the  Amor  School's  instructional  time  (Major  P.  Carroll,  DOTD ,  Personal  Com¬ 
munication,  20  May  1985)-  Major  Carroll  has  also  claimed  that  the  CBI  course¬ 
ware  will  be  designed  for  different  areas  of  armor  training,  including  advanced 
and  basic  training  courses. 


A  major  source  of  support  for  CBI  use  in  military  training  is  the  results 
of  research  studies  which  suggest  advantages  of  CBI  over  conventional 
instruction.  Several  reviews  of  the  CBI  literature  (e.g.,  Kearsley  et  al . , 
1983;  Kemner-Richardson,  Lamos,  &  West,  1984;  Orlansky  &  String,  1981;  and 
Orlansky,  1983;  1985)  have  indicated  that  this  instructional  medium  has  led 
to:  (a)  reduced  training  time  and  (b)  reduced  instructional  costs.  These 
reports  have  also  indicated  that  students  seemed  to  favor  taking  a  course  by 
means  of  CBI  rather  than  receive  conventional  instruction.  The  CBI  litera¬ 
ture  for  civilian  training  also  suggested  that  CBI  training  would  be  the  most 
effective  instructional  mode  for  helping  students  to  achieve  a  better  mastery 
of  the  instructional  content  and  for  dealing  with  individual  learning  differ¬ 
ences.  The  robustness  of  these  findings  for  both  civilian  and  military 
training  will  be  analyzed  in  this  paper. 


A  major  weakness  in  the  CBI  literature  is  that  very  few  reviews  have 
dealt  with  analyzing  the  possible  methodological  problems  in  this  research 
area.  This  paper  will  explore  the  extent  to  which  previous  research  has 
confounded  media  effects  with  extraneous  treatment  effects.  Avner  (1978; 
1979)  and  Clark  (1985a)  have  noted  that  CBI  is  a  medium  rather  than  an  in¬ 
structional  treatment  and  thus  researchers  must  determine  the  unique  educa¬ 
tional  contribution  inherent  in  CBI  as  a  primary  instructional  medium.  As 
Avner  (1979),  Director  of  Evaluation  at  the  Computer-Based  Educational  Re¬ 
search  Laboratory  at  the  University  of  Illinois  (CEREL),  has  stated,  "Com¬ 
puter-based  education  (another  term  for  CBI)  is,  despite  its  title,  more  a 
medium  for  communication  of  information  than  a  coherent  instructional  plan  or 
approach."  (pi 36 ) . 


Without  a  thorough  analysis  of  the  evidence  on  CBI  as  an  instructional 
medium,  one  is  ill-equipped  to  make  recommendations  and  decisions  about  the 
use  of  CBI  as  a  primary  instructional  medium  for  military  training.  Military 
educators  must  know  if  the  courseware  associated  with  the  CBI  system  can  be 
more  or  less  effectively  delivered  by  other  instructional  media.  This  review 
will  then  examine  the  state  of  civilian  and  military  CBI  research  on  costs, 
learning  achievements,  individual  differences,  and  student  attitudes.  These 
four  main  areas  of  CBI  research  will  be  discussed  in  the  next  four  sections, 
respectively.  Methodology  concerns  in  CBI  research  and  implications  for 
future  CBI  research  will  also  be  discussed.  The  civilian  CBI  research  is 
discussed  because  decisions  about  using  CBI  in  military  training  have  been 
based  upon  this  research  literature. 


One  must  also  carefully  examine  the  CBI  literature  because  history  has 
shown  that  educational  innovations  which  were  implemented  without  sufficient 


research  and  planning  were  always  abandoned  for  still  newer  educational  inno¬ 
vations  (Montague  &  Wulfeck,  1984).  Ragsdale  (1982)  has  suggested  that  the 
history  of  CBI  may  be  following  the  same  path  as  other  recent  educational 
innovations,  e.g.,  programmed  instruction,  from  initial  enthusiasm  and 
excitement  for  the  innovation  to  its  eventual  abandonment.  With  the  sizeable 
financial  investment  associated  with  large-scale  CBI  implementation,  the 
military  can  ill  afford  to  implement  and  then  abandon  this  instructional 
medium  because  of  poor  research  and  planning. 
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Costs 


A  major  obstacle  to  widespread  CBI  implementation  is  the  extensive 
capital  investment  associated  with  acquiring  a  system.  Reed  (1983)  cited  an 
EDUCCM  Bulletin  (1983)  report  which  estimated  that  institutions  should  expect 
to  pay  between  $1,000  and  $6,000  per  student  for  a  1985-1990  CBI  system. 
Hofstetter  (1983)  has  noted  that  the  per  terminal  hardware  costs  for  a  PLATO 
(Programmed  Logic  for  Automatic  Teaching  Operations)  system  is  $2,900.  Avner 
(1986)  has  noted,  however,  that  hardware  costs  are  substantially  lower  in 
1986  than  in  1982  from  a  $128,000  yearly  investment  to  a  $33*760  investment 
for  the  terminal  hardware  associated  with  a  standard  32  terminal  PLATO  sys¬ 
tem.  However,  other  costs  associated  with  the  PLATO  system,  e.g.,  costs  on 
conmunication  systems,  have  doubled  and  tripled  in  the  last  four  years.  He 
has  concluded  that  an  institution  can  expect  a  yearly  investment  of  $135,000- 
$150,000  for  an  entire  system.  These  costs  reflect  the  charges  from  a  non¬ 
profit  organization  (CEREL)  and  for  the  following  items:  1)  thirty-two  termi¬ 
nals;  2)  power  for  terminals;  3)  terminal  site;  4)  disk  storage  space;  5) 
system  interface;  6)  central  computer;  7)  system  software;  8)  central  build¬ 
ing;  9)  service  travel;  10)  support  personnel;  11)  support  equipment;  12) 
supplies  and  spare  parts;  and  13)  communication  system  to  host  computer. 
Avner' s  figures  do  not  reflect  expenditures  for  courseware  materials  nor  for 
instructors. 

Hardware  costs  for  MicroTICCIT  (Microcomputer  Network  of  the  Time-Shared 
Interactive  Computer-  Controlled  Information  Television  System)  are  approxi¬ 
mately  $250, 000  for  a  ten  terminal  system.  This  price  estimation  includes  a 
host  terminal,  color  terminals,  videodisc  players,  and  a  hefty  charge  for 
lifetime  authoring  privileges.  These  costs  do  not  reflect  expenditures  for 
producing  courseware  materials  nor  for  classroom  personnel . 

Christiansen  (1985)  observed,  however,  that  the  cost  value  of  any  product 
should  be  determined  by  its  life  cycle  costs  rather  than  by  its  initial  out¬ 
lay  costs.  As  indicated  previously,  many  CBI  reports  (e.g.,  Hofstetter, 

1983;  Kemner-Richardson  et  al.,  1984;  Orlansky,  1985;  Orlansky  &  String, 

1983;  and  Seidel,  1980)  have  suggested  that  such  systems  are  cost  effective 
(same  effectiveness  at  reduced  costs  or  more  effective  at  the  same  cost  than 
alternative  media)  and  would  eventually  lead  to  savings  in  educational  expen¬ 
ditures.  Kemner-Richardson  et  al .  noted,  for  example,  that  the  PLATO  system 
as  compared  to  conventional  instruction  would  lead  to  a  $180,000  a  year  sav¬ 
ings  for  an  average  military  educational  program.  Such  analyses  of  life 
cycle  cost  savings  attributable  to  a  CBI  system  traditionally  have  involved 
estimating  operating  costs  and  personnel  costs  which  would  accompany  imple¬ 
mentation  of  new  programs.  Life  cycle  cost  estimates  for  military  systems 
have  also  involved  calculating  the  dollars  saved  by  reducing  military  train¬ 
ing  time. 
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Training  Time 


A  most  pervasive  finding  in  the  CBI  literature,  especially  the  military 
literature,  is  that  use  of  CBI  leads  to  reduced  training  time.  Orlansky 
noted  that  the  median  savings  time  for  19  military  studies  when  compared  to 
conventional  instruction  was  30  percent  with  a  time  savings  range  of  -31  to 
89  percent.  These  studies  have  also  failed  to  find  any  noticeable  decrements 
in  students'  learning  associated  with  spending  less  time  in  instruction. 

Dallman,  Deleo,  Main,  and  Gillman  (1977)  reported  that  such  a  30J  time 
reduction  would  bring  about  a  yearly  savings  of  $23*000  for  an  Army  course  on 
vehicle  repair  which  trains  approximately  375  students  per  year.  Orlansky 
(1985)  believed  that  a  30Jt  time  savings  could  translate  to  a  yearly  financial 
savings  as  large  as  $10  million  in  FY  1977  money  for  training  50,000  Navy 
students.  Both  Dallman  et  al’s  and  Orlansky's  cost  estimates  are  based  upon 
calculating  students'  pay  and  allowances  for  the  amounts  of  time  saved  in 
training. 

Training  time  savings  could  further  reduce  military  expenditures  by 
allowing  more  students  to  complete  a  course  without  the  military's  having  to 
hire  additional  instructors.  A  three  week  savings  in  a  ten-week  course  would 
allow  the  military  to  teach  the  same  course  a  few  more  times  each  year  with¬ 
out  hiring  additional  staff.  Dallman  et  al.'s  (1977)  and  Orlansky's  (1985) 
cost  savings  estimates  may  then  be  conservative. 

CBI  and  conventional  instructions,  however,  are  not  equivalent  media  for 
measuring  students'  training  time.  Avner,  Moore,  and  Staith'3  (1980)  argument 
that  CBI  should  be  compared  to  other  self-pace d  individualized  instructional 
media,  e.g.,  programmed  texts,  is  especially  relevant  for  training  time  data. 
CBI  and  other  forms  of  individualized  instruction  allow  the  students  to  pro¬ 
ceed  at  their  own  pace,  while  conventional  instruction  does  not. 

Mixed  results  have  been  found  when  CBI  has  been  compared  to  programmed 
texts.  Time  savings  of  approximately  50  percent  for  both  CBI  and  programmed 
text3  as  compared  to  conventional  instruction  have  in  fact  been  reported  when 
all  three  media  have  simultaneously  been  tested  (Orlansky,  1985).  Dallman  et 
al.  (1977)  cited  a  study  conducted  at  the  Aberdeen  Proving  Grounds  which 
showed  a  1 0%  savings  time  for  CBI  when  directly  compared  to  programmed  texts. 
Other  studies  (e.g.,  Dallman  et  al.,  and  Hemphill,  1986)  found,  however,  that 
students  took  significantly  longer  to  complete  a  CBI  lesson  than  to  complete 
an  equivalent  programmed  text  lesson.  Time  savings  may  then  not  be  ascrib- 
able  to  CBI,  but  to  self-pacing,  which  characterizes  nearly  all  modern 
instructional  innovations.  Programmed  texts  are  usually  a  fraction  of  CBI's 
cost  for  initial  implementation.  According  to  the  Command  and  Staff  Depart¬ 
ment  at  Fort  Knox,  the  production  costs  for  a  133  page  programmed  text  are 
between  $1.00  and  $3*00  depending  on  whether  the  book  is  published  by  con¬ 
tract  or  by  the  military. 

Another  problem  with  the  CBI  literature  on  time  savings  is  that  CMI  has 
been  compared  to  conventional  instruction.  These  time  savings  effects  can 
only  be  attributable  to  differences  in  instructional  management  techniques 
e.g.,  time  in  scoring  tests,  rather  than  instructional  attributes  inherent  in 
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CBI.  Evaluators  should  then  test  the  effects  of  CMI  with  other  (perhaps  less 
costly)  administrative  aids.  Again,  confounded  comparisons  have  been  made 
in  measuring  time  savings  associated  with  CBI. 

A  third  problem  with  the  time  savings  data  is  a  lack  of  replicated  find¬ 
ings.  As  stated  by  Longo  (1972): 

A  paramount  notion  underlying  any  research  is  that  of  replica¬ 
tion.  In  question,  of  course,  is  the  matter  of  reliability  of 

information  across  time,  samples,  and  conditions.  (p38) 

Without  replication  and  information  on  reliability,  research  data  have  little 
value.  Qrlansky's  (1985)  report  on  the  time-savings  data  does  not  include 
any  description  of  studies  which  were  replicated  for  similar  instructional 
programs  using  the  same  computer  system.  Also  inconsistent  findings  were 
reported  by  Orlansky  when  CBI,  regardless  of  the  computer  system,  and  conven¬ 
tional  instruction  were  compared  for  similar  instructional  situations.  For 
example,  time  savings  for  electronic  courses  associated  with  CBI  were  found 
for  Army  personnel  at  the  Signal  Training  School  but  not  for  Navy  personnel 
at  San  Diego. 

Operating  Costs 

Determining  life  cycle  costs  also  includes  estimating  the  hidden  costs 
associated  with  operating  a  CBI  system.  Kopstein  and  Seidel  (1969)  have 
identified  maintenance  and  courseware  production  costs  as  the  major  variables 
in  estimating  hidden  costs. 

Hofstetter  (1983)  noted  that  the  maintenance  costs  of  the  PLATO  system 
would  be  minimal .  He  estimated  the  hourly  maintenance  cost  per  PLATO  system 
to  be  21  cents.  This  figure  is  based  upon  dividing  the  number  of  terminals 
into  the  money  included  in  the  contract  for  maintenance  costs.  His  calcula¬ 
tions  do  not  consider  the  system's  actual  reliability. 

System  reliability  is  important  to  measure,  because  repairing  malfunc¬ 
tions  will  cost  additional  money  in  repairing  the  problem  areas  and  in  train¬ 
ing  time  losses.  Holmes  (1982)  has  indicated  that  an  unreliable  system  may 
result  in  severe  loss  of  instructional  time  as  faulty  equipment  may  have  to 
be  shipped  to  outside  repair  centers.  A  system  reliability  index — (failures 
per  hour  X  terminals  affected) /(working  days  by  terminals  affected)  X  100 — 
has  been  developed  by  Francis,  Welling,  and  Levy  (1983)* 

Avner  (1986)  has  developed  another  measure,  PI  =  (#  interruptions  +  mean 
length  of  interruptions)/ total  time)  or  PI  =  (1  +  mean  length  of  interrup- 
tions)/(mean  time  between  interruptions).  Avner  has  claimed  that  these  PI 
indexes  are  the  most  sensitive  measure  of  system  reliability  because  they 
reflect  the  disruptive  effect  of  any  interruption  upon  the  instructional 
program.  PI  can  also  be  computed  for  each  component  of  a  system  (e.g., 
videodisc  failures)  and  the  unreliability  of  the  different  components  can  be 
added  to  provided  an  overall  system  reliability  index. 
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Such  reliability  data  for  a  CBI  system  must  be  compared  to  similar  data 
found  for  alternative  programs  (Boldovici  &  Scott,  1984),  otherwise  the  data 
are  meaningless.  Evaluators  should  obtain  information  about  delays  in  con¬ 
ventional  instruction.  Seidel  (1980)  suggested  that  such  information  for 
non-CBI  programs  has  rarely  been  ascertained. 

Seidel  (1980)  has  also  suggested  that  cost  comparisons,  e.g.,  determining 
a  system's  reliability,  between  prototype  CBI  systems  and  fully  operational 
educational  systems  are  not  a  fair  test  of  CBI's  potential  cost  value.  CBI 
evaluations  are  usually  conducted  for  a  prototype  or  a  newly  implemented 
system,  such  as  Francis  et  al.'s  evaluation.  Such  evaluations  may  then  not 
be  an  accurate  assessment  of  a  fully  fielded  system's  reliability  and  corre¬ 
sponding  costs.  Evaluations  are  needed  in  which  reliability  data  are  col¬ 
lected  during  the  initial  assessment  period  and  after  a  few  years  of 
operation.  Information  from  these  evaluations  would  help  CBI  professionals 
to  more  clearly  understand  the  discrepancies  found  in  a  system's  reliability 
between  initial  evaluation  and  actual  use. 

A  related  question  to  system  reliability  is  how  long  a  system  can  last. 
Avner's  yearly  projections  for  the  PLATO  system  cited  at  the  beginning  of  the 
costs  section  were  based  upon  a  ten-year  life-span.  Except  for  Avner's  data, 
little  is  known  about  the  average  life-span  for  a  CBI  system.  Accurate  in¬ 
formation  about  a  system's  potential  life-span  is  needed  to  better  understand 
the  life-cycle  costs  associated  with  CBI. 

Production  cost  differences  between  CBI  and  other  instructional  media 
have  also  rarely  been  systematically  analyzed.  Several  CBI  proponents  (e.g., 
Avner,  1982;  Burnside,  1985;  and  Dallman  et  al,  1977)  suggested  that  long¬ 
term  production  costs  might  be  cheaper  for  CBI  than  for  other  instructional 
media.  Their  claims  have  been  based  on  the  assumption  that  updating  CBI 
courseware  is  relatively  easy  and  inexpensive.  Dallman  et  al .  have  argued, 
for  example,  that  PLATO'S  editing  features  help  to  reduce  sharply  the  costs 
involved  in  updating  and  modifying  CBI  lessons.  According  to  Avner,  updating 
PLATO  lessons  is  a  function  of  the  quality  of  the  original  work  and  the 
volatility  of  the  subject  matter.  Some  PLATO  lessons  need  few  modifications 
after  eight  years  of  continuous  use.  Another  reason  for  reduced  long-term 
production  costs  is  that  the  CBI  system  will  allow  the  same  instructional 
materials  to  be  repeatedly  presented  throughout  the  system's  lifetime,  while 
some  separate  programmed  texts  may  have  to  be  created  for  each  student.  Very 
few  CBI  evaluations  have  examined  the  differences  in  actual  expenses  (and 
problems)  with  updating  the  instructional  materials  for  CBI  and  for  other 
instructional  media  (Orlansky,  1985). 

Initial  developmental  costs  for  a  CBI  program  might  be  quite  expensive. 
Walker  (1984)  reported  that  the  developmental  costs  for  a  CBI  program  might 
range  between  $2,000  and  $100,000  per  contact  hour.  Johnson  and  Plake  ( 1 980 ) 
noted  that  it  would  have  cost  the  University  of  Nebraska  library  over  $30,000 
to  present  a  one-hour  program  to  4,000  students.  By  contrast,  similar  text 
materials  cost  a  few  hundred  dollars  to  produce  (Walker,  1984).  These  expen¬ 
ditures  associated  with  initial  courseware  development  might  then  negate  the 
previously  discussed  savings  associated  with  CBI  for  other  production  costs. 


An  important  consideration  in  determining  cost-effectiveness  is  the 
hourly  cost  associated  with  using  the  instructional  medium.  Tremendous 
variation  exists  in  the  hourly  cost  of  CBI  service.  Avner  (1982)  has  noted 
that  PLATO  terminal  costs  can  vary  depending  on  terminal  use,  software  use, 
and  distance  from  the  central  computer  from  $1.76  per  terminal  to  $10.94  per 
terminal.  Okey  and  Majer  (1976)  have  noted  that  CAI  costs  per  student  con¬ 
tact  hour  are  $2.07.  The  $1.76  and  $2.07  figures  represents  nearly  continu¬ 
ous  and  daily  use  of  the  system  while  the  $10.94  figure  represents  sporadic 
CBI  use.  None  of  the  figures  cited  has  included  the  expenditures  set  aside 
for  instructors.  Also,  comparisons  of  hourly  costs  for  using  CBI  have  not 
been  made  with  those  for  other  media. 

Himwich  (1977)  and  Stone  (1985)  have  detailed  the  financial  costs 
involved  in  developing  in-house  courseware.  Himwich  noted  that  military 
courseware  developers  must  be  formally  trained  in  computer  procedures  and 
educational  practices.  He  also  found  that  it  took  around  230  hours  to  com¬ 
plete  one  student  contact  hour  of  courseware.  Stone's  developmental  team 
consisted  of  five  Army  Captains,  civilian  educational  specialists,  and  out¬ 
side  consultants.  These  Army  personnel  also  had  to  take  a  computer  course  in 
order  to  design  this  courseware.  Himwich' s  project  was  aborted  because  of 
financial  problems  and  Stone's  team  did  not  meet  its  goal  of  completing  100 
hours  of  instructional  time  by  the  following  year. 

Several  instructional  specialists  (e.g..  Holmes,  1982;  and  Montague, 
Wulfeck,  &  Ellis,  1983)  believe  that  in-house  courseware  can  be  developed 
rapidly  and  successfully.  Holmes  has  argued  that  a  template  system  would 
allow  instructors  to  design  CBI  courseware  in  a  relatively  short  space  of 
time  and  with  little  knowledge  of  computer  programming.  Montague  et  al.  have 
argued  that  computer-aids  can  help  reduce  the  time  and  thus  costs  involved  in 
producing  in-house  courseware.  Cost-effectiveness  of  such  templates  and 
computer-aids  has  rarely  been  studied.. 

Avner,  3nith,  &  Tenczar  (1984)  did  complete  a  longitudinal  observation  of 
143  independent  production  groups.  They  found  that  production  efficiency  was 
best  predicted  by  :  (a)  having  a  strict  deadline  time;  (b)  using  software 

authoring  tools;  (c)  having  subject  matter  expertise;  and  (d)  having  experi¬ 
ence  with  media  and  individualized  instruction  methods.  The  use  of  software 
authoring  tools,  which  can  be  difficult  for  the  inexperienced  programmer, 
provides  the  most  cost-effective  method  of  improving  both  production  effi¬ 
ciency  and  quality.  Avner  et  al.  have  suggested  the  use  of  EnBASIC  to  meet 
the  needs  of  most  courseware  programmers  working  with  the  Apple  II  series 
computers  and  other  computers  compatible  with  this  series.  Other  systems, 
e.g.,  MICROTICCIT,  however,  dictate  that  a  special  authoring  language  be 
used .  Thus  a  successful  courseware  programmer  might  have  to  know  several 
authoring  languages  or  programs  for  just  one  type  of  system. 

Avner  (1979;  1982)  also  emphasized  the  importance  of  the  cousreware  de¬ 
signers'  having  previous  experience  with  the  medium  and  the  authoring  lan¬ 
guage.  Avner  (1979)  noted  that  an  inexperienced  team  of  courseware  designers 
(less  than  a  year  of  experience)  took  between  165  to  610  hours  to  generate 


one  student  contact  hour  of  instruction  while  highly  experienced  designers 
(greater  than  two  years  of  experience)  took  between  27  to  180  hours  to  gener¬ 
ate  a  similiar  amount  of  courseware.  He  then  concluded  that: 

Rich  of  the  published  data  (e.g..  Stone's)  on  production  costs 
for  CBE,  for  example,  are  based  on  groups  with  only  about  a  year 
of  working  experience  with  the  medium.  Such  data  may  grossly 
overestimate  production  costs  possible  in  steady-state  (exist¬ 
ing)  operations  by  experienced  design  teams  and  underestimate 
start-up  costs  (for  inexperienced  designers),  (pi 37) 

Avner  (1982),  however,  has  noted  that  430  hours  of  experience  with  the 
TUTOR  (the  PLATO)  authoring  language  is  needed  for  experienced  college  teach¬ 
ers  to  produce  their  own  administrative  and  pedagogical  structures.  He  also 
noted  that  $1000  is  the  average  cost  for  an  experienced  university  staff  to 
program  an  hour's  worth  of  CBI  material.  This  $1000  figure  does  not  repre¬ 
sent  the  overhead  expenditures  found  for  courseware  developed  by  private 
companies  and  this  figure  may  be  reduced  when  authoring  tools  and  computer- 
aid  materials  are  further  refined.  CBI  production  costs  must  be  figured  for 
both  "steady-state"  and  "start-up"  costs  and  must  be  compared  with  the  same 
costs  involved  in  developing  courseware  for  other  media,  e.g.,  device  simula¬ 
tors. 

Personnel  Costs 


Personnel  costs  usually  are  determined  by  estimating  the  increases  or 
decreases  in  the  number  of  educational  personnel  which  would  accompany  imple¬ 
mentation  of  new  programs.  As  previously  argued,  training  time  savings  could 
allow  the  military  to  train  more  students  without  hiring  more  instructors. 
Assuming  that  the  yearly  expenditures  for  civilian  and  military  instructors, 
including  benefits,  range  between  $20,000  to  $30,000,  then  the  resulting  cost 
savings  for  not  hiring  instructors  could  be  in  the  millions  of  dollars. 

Also,  if  CBI  is  indeed  more  effective  than  conventional  instruction,  then, 
ultimately,  teachers  may  need  not  be  the  primary  instructional  medium. 
Teachers  would  only  be  necessary  to  tutor  problematic  students  and  monitor 
operations  of  the  computer  system.  Again,  more  students  could  be  taught  by 
fewer  instructors. 

Burnside  (1985)  has  argued  that  CBI  can  reduce  the  educational  support 
staff.  His  argument  is  based  upon  the  expectation  made  by  computer  experts 
(e.g..  Baker,  1978)  that  CBI,  especially  CMI,  can  accomplish  the  administra¬ 
tive  tasks  usually  done  by  the  clerical  staff.  Dallman  et  al.  (1977)  also 
projected  that  support  staff  needs  would  require  minimal  expenditures.  They 
estimated  spending  only  $7,200  a  year  for  support  personnel's  operational 
maintenance  tasks.  This  figure  was  based  upon  three  civilians — a  GS-9,  GS-7, 
and  GS-5 — spending  20%  of  their  time  on  such  problems.  Dallman  et  al.'s 
projection  may  be  conservative  as  maintenance  tasks  may  occupy  more  of  these 
people's  time  than  anticipated.  Also,  neither  Burnside  nor  Dallman  et  al . 
made  any  mention  of  the  costs  involved  in  hiring  support  staff  to  operate  and 
update  the  system.  Avner  (1982)  has  noted  that  the  total  overhead  for  sup¬ 
port  personnel  for  a  civilian  32-terminal  PLATO  system  is  $989  (at  1982  mar¬ 
ket  prices)  per  terminal.  This  figure  includes  five  full-time  administration 


positions,  nine  full-time  maintenance  positions;  seven  full-time  operator 
positions;  and  fifteen  fall-time  user  service  (proctor)  positions.  Burn¬ 
side's  and  Dallman  et  al.'s  position  may  then  not  reflect  all  support  staff 
costs  while  Avner's  data  may  be  a  gross  overestimation  for  a  military  CBI 
implementation.  Uh fortunately,  this  author  could  not  find  any  study  which 
provided  information  about  actual  support  staff  expenditures  for  implemented 
CBI  systems  at  a  military  training  site.  Nor  was  any  study  found  which  com¬ 
pared  differences  in  support  staff  needs  for  CBI  with  those  of  other  media. 

Several  sources  (e.g.,  Bellinger,  1986;  Shavelson  et  al.,  198*1;  Walker, 
198*1;  and  Wisher  &  O'Harra,  1981)  have  indicated  that  CBI  may  not  reduce 
personnel  expenditures.  Wisher  and  O'Hara  reported  that  off-line  instructor 
task  elements  of  PREST  (Performance-Related  Enabling  Skills  Training)  CBI 
program  increased  rather  than  decreased  instructor  dependence.  They  also 
failed  to  find  any  significant  performance  differences  between  PREST  subjects 
and  those  who  were  instructed  by  conventional  courseware.  They  concluded 
that  PREST  was  not  as  cost-efficient  as  was  the  conventional  system. 

Bellinger  (1986);  Shavelson  et  al.  (1984);  and  Walker  (1984)  have  sug¬ 
gested  that  personnel  costs  should  also  include  expenses  associated  with 
training  teachers  to  be  "computer-experts."  Shavelson  et  al.  observed  that 
using  a  CBI  system  in  a  typical  school  necessitated  a  staff  development  pro¬ 
gram  which  trained  teachers  to  be  thoroughly  knowledgeable  about  computers. 
Walker  noted  that  experiences  at  the  Microcomputer  Institute  for  Educators  at 
Stanford  University  demonstrated  that  180  hours  of  study  (eight  hours  a  day 
for  a  month)  was  needed  to  train  highly  motivated  teachers  to  use  micro¬ 
computers  properly  in  the  classroom.  One  hundred  eighty  hours  of  study  is 
not  excessive  for  training  In  "computer  expertise"  as  the  ADAPT  course 
(authoring  courseware  for  MicroTICCIT)  involves  nearly  three  weeks  of  train¬ 
ing  at  eight  hours  per  day.  Walker  also  noted  that  civilian  schools  do  not 
have  the  resources  to  provide  the  needed  additional  inservice  training  for 
ever-changing  computer  systems.  The  Army,  though,  with  its  computer  literacy 
program  may  be  better  able  to  provide  its  instructors  with  this  additional 
training.  Clearly,  such  hidden  costs  as  these  teacher  training  programs  must 
be  included  in  any  analysis  of  CBI's  life-cycle  costs.  Of  course,  possible 
hidden  costs  in  personnel  expenditures  for  conventional  instruction  must  also 
be  examined. 

Conclusions 

Any  claims  about  the  comparative  costs  of  CBI  and  other  educational  media 
are  premature.  As  discussed,  very  few  studies  have  actually  compared  the 
cost-effectiveness  of  a  CBI  system  with  the  cost-effectiveness  for  other 
media.  Current  CBI  cost  estimates  may  have  also  not  included  all  costs  for 
thi3  medium  and  may  have  overestimated  certain  costs. 

Claims  about  the  cost  savings  associated  with  using  a  CBI  system  may  also 
be  premature.  Training  time  savings  may  not  be  attributable  to  this  medium, 
but  to  self-pacing  which  characterizes  any  individualized  instructional 
program.  Conclusions  about  training  time  reductions  and  CBI  are  problematic 


because  information  is  not  readily  available  about  the  reliability  of  these 
data.  Also  the  cost  savings  in  personnel  expenditures  associated  with  CBI 
have  not  been  demonstrated. 

The  following  methodological  concerns  must  then  be  incorporated  into 
future  cost  comparisons.  If  training  time  is  the  important  criterion  meas¬ 
ure,  than  CBI  must  be  compared  with  other  self-paced  instructional  systems. 
Information  about  costs  must  be  gathered  for  implemented  systems.  Seidel 
(1980)  has  noted  that  reliable  and  meaningful  cost  information  will  only  be 
obtained  when  the  innovation  is  in  a  stable,  operational  state.  All  poten¬ 
tial  costs  for  the  CBI  and  the  alternative  medium  must  also  be  identified  and 
examined.  Cost  analyses  must  be  done  fen  eaeh  -  JU- Avner  .  - 

(1978)  stated,  "Just  because  20  prestigious  researchers  found  CBE  (CBI)  to  be 
cost-effective  in  their  application  does  not  mean  it  will  automatically  be 
cost-effective  in  yours."  (p25) 


Learning  Achievements 


As  previously  indicated  a  major  assumption  behind  implementing  CB1  is 
this  medium's  positive  impact  upon  students'  learning  processes.  Many  CBI 
professionals  (e.g..  Banger t-Drowns,  Kulik  &  Kulik,  1985;  Eberts  &  Brock, 

1 98^4 ;  Fletcher  &  Rockway,  1986;  Kearsley  et  al.,  1983;  Kulik,  Kulik,  &  Cohen, 
1980;  Robinson  &  Kirk,  1984;  and  Suppes  &  Morningstar,  1969)  have  indicated 
that  CBI  has  or  will  lead  to  the  development  of  superior  instructional  mate¬ 
rials.  Kearsley  et  al,  stated  that,  "We  (CBI  professionals)  have  ample  evi¬ 
dence  that  computers  can  be  used  to  make  instruction  more  effective  or 
efficient  in  a  variety  of  ways."(pl4) 

Critics  of  CBI  research  (e.g.,  Clark,  1985a  and  1985b;  and  Reinking, 

1985)  have  questioned  the  evidence  of  CBI's  instructional  superiority. 
Reinking  noted  that  comparisons  between  CBI  and  alternative  media  have  pro¬ 
duced  inconsistent  findings.  These  critics  have  also  questioned  the  implied 
assumption  that  CBI  is  an  inherently  superior  instructional  medium. 

An  examination  of  the  CBI  training  studies  is  thus  needed  to  help  resolve 
this  dispute.  Existent  training  studies  have  usually  compared  differences 
between  CBI  students  and  conventional  instruction  students  with  regard  to 
test  performance  and  course  completion.  This  latter  measure  concerns  the 
number  of  students  completing  the  course.  Such  studies  have  been  referred  to 
as  outcome  studies  (Kulik  et  al.,  1985).  Analysis  is  also  needed  of  CBI's 
relative  efficiency  for  facilitating  transfer.  Specific  CBI  features  affect¬ 
ing  students’  learning  will  be  discussed  in  this  section. 

Outcome  Studies 


Fletcher  and  Rockway  (1986)  have  suggested  that  CBI  is  an  efficient  in¬ 
structional  medium  for  military  training  purposes.  They  cited  evaluations 
conducted  by  each  service  demonstrating  CBI’s  potential  effectiveness.  Eval¬ 
uation  of  a  CBI  program  for  an  Army  Signal  Training  Course  showed  that  the 
CBI  group  achieved  higher  test  scores  and  less  attrition  than  did  the  control 
group  (Longo,  1972). 

Orlansky  (1985)  and  Orlansky  and  String  (1983)  have  provided  evidence 
challenging  CBI's  instructional  effectiveness  for  military  training.  Learn¬ 
ing  achievement  was  found  to  be  about  the  same  for  the  CBI  and  conventional 
instruction  students  in  32  of  the  48  military  cases  reviewed  by  Orlansky  and 
String.  Fifteen  of  these  studies  reported  slightly  superior  learning 
achievement  for  the  CBI  programs  while  one  study  reported  slightly  poorer 
achievement  for  the  CBI  group. 

Francis  et  al.  (1983)  provided  another  example  of  the  inconsistency 
found  in  CBI  results.  They  conducted  two  studies  of  CBI's  effectiveness  for 
training  at  the  US  Army  Missile  and  Munitions  Center  and  School.  PLATO  simu¬ 
lators  were  found  to  be  an  effective  and  efficient  instructional  medium  for 
the  TOW  (missile)  Field  Test  Site  Training  Course.  Data  analyses  failed. 
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however,  to  find  any  differences  between  PLATO  students'  and  non-PLATO  stu¬ 
dents'  test  results  for  the  HAWK  Continuous  Wave  Radar  Repair  Training 
Course . 

Montague  (1984)  and  Orlansky  (1985)  claimed  that  consistent  evidence  has 
been  found  in  the  civilian  training  literature  for  CBI's  effectiveness.  This 
conclusion  is  based  upon  a  series  of  meta-analyses  (Bangert- Drowns  et  al., 
1980;  Kulik,  Bangert,  &  Williams,  1983;  and  Kulik  et  al.,  1980)  showing  no¬ 
ticeable  effects  for  CBI  upon  students'  test  scores.  Bangert-Drowns  et  al. 
found  that  exam  scores  for  secondary  school  students  in  a  CBI  program  were  .4 
standard  deviations  higher  than  were  the  scores  reported  for  the  conventional 
instruction  students.  This  finding  suggested  that  CBI  was  moderately  more 
effective  than  were  the  conventional  instruction  programs. 

Inconsistent  data  for  CBI's  effectiveness  are  also  found  in  the  civilian 
training  literature.  A  close  examination  of  Kulik  et  al.,'s  (1980)  data  has 
indicated  that  only  one-quarter  of  the  54  studies  cited  found  large  or  medium 
effects  for  CBI  upon  college  students'  exam  performance.  Alderman,  Appel, 
and  Murphy's  (1978)  often  cited  evaluation  of  PLATO'S  and  TICCIT's  effective¬ 
ness  for  community  college  students  also  found  inconsistent  results.  The 
TICCIT  programs  had  a  significantly  positive  impact  upon  students'  math 
achievement.  However,  significantly  fewer  students  completed  the  math  course 
for  the  TICCIT  program  than  completed  the  conventional  instruction  program. 

No  significant  differences  were  found  between  PLATO  and  conventional  instruc¬ 
tion  students'  test  scores  and  completion  rates.  These  inconsistent  findings 
may  be  due  to  several  factors  such  as  courseware  differences  or  problematic 
research  practices. 

Methodological  Criticisms 

Clark  and  his  associates  (Clark,  1983;  Clark,  1985a  and  1985b;  Clark  & 
Leonard,  1985;  and  Clark  &  Salomon,  1985)  have  suggested  that  the  cited  in¬ 
consistent  findings  in  the  CBI  literature  are  due  to  problematic  research 
practices.  They  have  suggested  that  CBI  has  only  been  shown  to  be  more  ef¬ 
fective  than  conventional  instructional  programs  when  confounded  comparisons 
have  been  made  between  CBI  and  other  media.  They  have  also  indicated  that 
such  confounded  comparisons  are  prevalent  in  CBI  effectiveness  literature. 
Clark  and  Leonard  noted  that  75)1  of  the  studies  which  were  randomly  selected 
for  their  review  had  serious  design  flaws.  And  Clark  (1985b)  has  noted  that 
the  most  probable  source  of  confounding  in  CBI  research  has  been  that  differ¬ 
ent  instructional  treatments  were  employed  in  the  computer  and  control  condi¬ 
tions. 

Failure  to  match  instructional  content  is  a  common  problem  in  the  CBI 
literature.  Alderman's  (1974)  plans  for  the  evaluations  of  TICCIT  and 
PLATO  described  methods  for  comparing  students'  achievement  for  dissimilar 
instructional  materials.  He  has  claimed  that  such  comparisons  can  be  made 
when  substantial  overlap  exists  between  objectives  for  the  different  courses. 
Some  CBI  studies,  e.g..  Wisher  and  O'Hara  (1981)  and  those  cited  by  Fletcher 
and  Rockway  (1986)  may  have  indeed  compared  similar  objectives  with  substan¬ 
tially  different  instructional  content.  The  CBI  and  conventional  instruction 


programs  in  Wisher  and  O'Hara's  study  were  designed  to  remedy  reading  prob¬ 
lems.  The  CBI  program  consisted  of  eleven  reading  modules  within  the  Basic 
Skills  Learning  System  devised  by  the  Control  Data  Corporation  while  the 
conventional  classroom  instruction  included  reading  materials  created  by 
Memphis  State  University.  The  similarity  of  the  instructional  content  for 
these  two  reading  programs  was  never  discussed.  Wisher  and  O'Hara's  inabil¬ 
ity  to  find  significant  differences  between  programs  might  be  due  to  differ¬ 
ences  in  instructional  content.  Conversely,  the  effects  for  CBI  reported  by 
Alderman  et  al.  (1978)  and  Fletcher  and  Rockway  might  have  been  attributed  to 
presenting  different  content  and  not  to  any  features  inherent  in  the  CBI 
program. 

Morrison  and  Witmer’s  (1983)  comparative  evaluation  of  computer-based  and 
print-based  job  aids  for  a  military  maintenance  task  is  one  of  the  few  CBI 
studies  which  has  matched  instructional  content.  They  made  a  special  effort 
to  make  the  wording  and  format  of  the  two  programs  identical.  These  two 
programs  did  differ  in  that  the  computer-based  program  used  a  branching  in¬ 
structional  format  while  the  print-based  program  used  a  linear  model.  Given 
these  differences,  the  positive  effects  found  for  the  computer-based  program 
might  be  due  to  either  the  medium  or  the  branching  sequence. 

Comparing  similar  instructional  materials  also  requires  that  identical 
efforts  be  made  in  designing  the  courseware.  Clark  ( 1 984 ;  1985a)  discussed 
the  possible  confounding  in  CBI  research  due  to  differences  in  designing  the 
courseware  for  CBI  and  conventional  instruction.  The  most  powerful  extrane¬ 
ous  factor  which  CBI  studies  rarely  control  for  is  that  a  greater  effort  has 
been  made  to  design  the  CBI  presentation  than  that  required  for  the  compara¬ 
tive  media  involved  (Clark,  1984).  Clark  does  not,  however,  provide  any 
evidence  that  instructional  effort  differences  do  exist  or  effect  CBI  re¬ 
search. 

Evidence  does  exist  that  some  CBI  program  development  has  involved  con¬ 
siderable  design  efforts.  As  previously  stated,  developing  CBI  courseware 
involves  considerable  time  and  effort.  Also  some  CBI  programs  (e.g., 

PLATO'S  and  TICCIT's)  were  designed  by  some  of  the  finest  educational  spe¬ 
cialists.  Bunderson  (1981)  noted  that  the  prototype  TICCIT  courseware  was 
primarily  a  function  of  the  analytical  theorems  and  artistic  talents  of  the 
courseware  development  team  at  Brigham  Young  University  with  some  help  from 
the  community  college  teachers.  The  Brigham  Young  staff  did  not  help  with 
the  development  of  any  of  the  conventional  instruction  courseware  used  in 
this  evaluation.  One  would  expect  that  efforts  associated  with  the  classroom 
instruction  design  did  not  approach  the  efforts  involved  in  the  TICCIT 
courseware  development. 

Several  CBI  professionals  (e.g..  Gray,  Pliske  &  Psotka,  1985;  Montague  & 
Wulfeck,  1984;  and  Walker,  1984)  challenged  the  assumption  that  considerable 
efforts  were  spent  developing  most  current  CBI  courseware.  In  fact.  Gray  et 
al.  have  suggested  that  most  current  CBI  courseware  is  underdeveloped.  Walker 
has  contended  that  there  is  a  lack  of  quality  CBI  courseware.  Furthermore, 
Stone’s  (1985)  and  Himwich's  (1978)  developmental  teams  may  have  spent  the 
majority  of  their  time  on  programming  and  not  on  courseware  designing. 


Clark's  (1984  and  1985a)  claims  about  confounding  due  to  instructional  ef¬ 
forts  differences  may  not  be  valid,  but  this  potential  confounding  must  be 
examined  in  future  CBI  studies. 

Clark  and  Leonard  (1985)  described  another  type  of  confounding  associated 
with  not  matching  instructional  content.  After  reviewing  42  randomly  select¬ 
ed  civilian  CBI  programs,  they  found  that  CBI  subjects  received  more  instruc¬ 
tion  in  completing  the  lessons  than  did  the  control  subjects.  One  example 
provided  by  Clark  and  Leonard  is  a  study  in  which  CBI  groups  have  received  as 
much  as  eight  minutes  more  of  drill-and-practice  time  than  have  the  conven¬ 
tional  instruction  groups.  Practice  effects  rather  than  the  medium  may  have 
influenced  the  achievement  gains  found  for  the  CBI  students.  The  extent  to 
which  such  additional  instructional  time  is  a  problem  depends,  of  course,  on 
the  observed  amount  of  instruction  to  the  amount  intended  by  a  system's  de¬ 
signer.  And  the  need  for  instructor  prompting  undermines  CBI's  promise  to 
reduce  costs  by  reducing  instructional  personnel.  Unwarranted  instruction, 
such  as  additional  prompting,  should  then  be  another  variable  measured  in  CBI 
research. 

Using  CBI  to  supplement  the  instructional  program  is  another  issue  asso¬ 
ciated  with  additional  instruction.  Several  studies  (e.g.,  Dengler,  1983; 
Jelden,  1985;  and  Mevarech  &  Rich,  1985)  have  indicated  that  CBI  is  effective 
when  used  with  a  conventional  instruction  program.  Most  of  these  studies, 
however,  did  not  include  a  comparative  group  receiving  either  a  similar 
amount  of  conventional  instruction  or  conventional  instruction  with  another 
educational  medium. 

Clark  (1983;  1985a)  and  Clark  A  Salomon  (1985)  have  argued  that  "novelty 
effects"  may  also  confound  CBI  research  findings.  They  claimed  that  the 
positive  instructional  effects  found  for  CBI  in  the  previously  cited  meta¬ 
analyses  were  due  to  students'  initial  motivation  and  enthusiasm  to  learn 
novel  courseware,  rather  than  to  any  characteristics  inherent  in  the  CBI 
system.  Clark  (1985a)  has  shown  that  CBI  effectiveness  may  be  very  different 
for  long-  and  short-term  studies.  Clark  noted  from  Kulik  et  al.'s  (1983) 
data  that  secondary  school  students'  achievement  scores  were  .56  standard 
deviations  higher  for  CBI  groups  than  for  conventional  instruction  groups 
when  the  studies  lasted  four  weeks  or  less.  Differences  between  groups' 
performances  reduced  to  .30  for  studies  lasting  five  to  eight  weeks  and  to 
.20  for  studies  lasting  more  than  eight  weeks. 

Kulik,  (Oulik,  and  Bangert-Drowns  (1985)  argued  that  "novelty  effects" 
were  not  pervasive  in  CBI  research.  They  reported  that  CBI  raised  students' 
achievement  scores  by  .26  standard  deviations  in  the  typical  long-term  evalu¬ 
ation  study  and  by  .34  standard  deviations  in  the  typical  short-term  study. 
These  differences  in  standard  deviations  are  not  statistically  significant. 
Since  short-  and  long-term  evaluations  were  not  defined  by  Kulik  et  al,  using 
Kulik  et  al.'s  data  to  refute  the  notion  of  "novelty  effects"  may  then  not  be 
valid.  Clark  (1985a)  conceded,  however,  that  "novelty  effects,"  were  not 
significant  detractors  from  the  validity  of  CBI  studies.  The  issue  of  nov¬ 
elty  effects  will  be  further  explored  in  the  section  on  student  attitudes. 
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Duration  problems  may  exist  for  military  training  studies.  Orlansky 
(1985)  and  Orlansky  and  String  (1983)  both  noted  that  half  of  all  CBI  mili¬ 
tary  courses  lasted  one  week  or  less.  Some  studies  (e.g.,  Francis  et  al.'s, 
1983)  involved  limited  comparisons  for  particular  course  segments.  Since 
most  military  training  lasts  for  a  few  months,  evaluation  of  less  than  a 
week's  duration  or  of  only  a  few  instructional  segments  are  not  sufficient 
for  making  equivalent  comparisons  between  CBI  and  other  modes  of  instruction. 
These  limited  durations  for  CBI  military  studies  may  work  against  the  possi¬ 
bility  of  finding  significant  differences  between  CBI  and  control  groups. 

Other  criticisms  of  outcome  studies  include  points  made  about  the  cost 
studies.  For  one  thing,  existent  outcome  studies  rarely  involved  an  imple¬ 
mented  system.  Several  CBI  proponents  (e.g.,  Montague  &  Wulfeck,  1984  and 
Gray,  1986)  have  argued  that  quality  CBI  programs  involves  quality  implemen¬ 
tation.  And  Bunderson  (1981)  noted  that  the  TICCIT  courseware  used  in  the 
Alderman  et  al.  (1978)  evaluation  was  not  completely  debugged.  Outcome  stud¬ 
ies  using  prototype  systems  may  then  provide  inaccurate  estimates  of  CBI's 
instructional  impact. 

Outcome  studies  have  also  rarely  been  replicated  and  have  rarely  involved 
comparisons  with  other  individualized  instructional  media.  Questions  then 
exist  about  the  stability  of  these  studies'  results.  And  any  effects  found 
for  CBI  may  then  not  be  due  to  the  media  but  rather  to  attributes  inherent  in 
individualized  instruction. 


Transfer  Studies 

Evaluating  CBI's  instructional  value  also  involves  determining  whether  a 
system  is  being  used  correctly.  Avner  (1978)  has  observed  that  the  correct 
usage  of  the  medium  is  more  important  than  the  fact  that  it  works.  Correct 
usage  of  any  educational  system  involves  helping  students  to  make  the  neces¬ 
sary  transfer  of  information  to  the  appropriate  environmental  situations. 

Controversy  exists  in  the  CBI  literature  regarding  this  medium's  effec¬ 
tiveness  for  facilitating  transfer.  Many  CBI  proponent's  (e.g.,  Gleason, 
1981;  Kern,  1985;  McLaughlin,  1985;  Montague,  1984;  Reigeluth  &  Garfield, 
1984;  and  Salomon,  1979)  have  suggested  that  CBI  would  help  develop  the 
learning  processes  needed  for  successful  transfer.  For  one  thing,  McLaughlin 
has  indicated  that  the  computer  is  ideally  suited  for  developing  both  proce¬ 
dural  and  declarative  level  knowledge .  Procedural  level  knowledge  is  a  basic 
understanding  of  the  steps  involved  in  completing  a  task  while  declarative 
level  knowledge  involves  an  understanding  of  the  principles  and  concepts 
associated  with  that  task  (Clark  &  Vogel,  1985).  According  to  McLaughlin, 

CBI  courseware  should  then  augment  students'  abilities  to  make  basic  proce¬ 
dural  level  transfer--e .g . ,  following  a  technical  manual  to  troubleshoot  an 
armor  system.  CBI  would  also  help  students  with  cognitive  transfer — e.g., 
understanding  the  principles  behind  the  repair. 

CBI  proponents  also  believe  that  this  medium  would  help  students  to  de¬ 
velop  the  memory  skills  needed  for  long-term  transfer.  One  reason  for  this 
belief  is  that  drill-and-practice  programs  would  enable  students  to  rehearse 
the  information  continuously.  Continuous  rehearsal  is  a  primary  way  of  stor- 
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ing  information  into  memory  (Landhauer,  1982).  Also,  many  computer  profes¬ 
sionals  and  behavioral  scientists  (e.g.,  Bunderson,  1981;  Kosslyn,  1981; 
Merrill  et  al.,  1980;  Olson  &  Bruner,  1974;  and  Salomon  &  Gardner,  1986) 
have  suggested  that  active  learning  and  dual-modality  processing  are  associ¬ 
ated  with  computer  instruction.  Active  learning  and  dual-modality  processing 
can  help  facilitate  deep  cognitive  processing  of  information.  And  informa¬ 
tion  processed  into  deep  cognitive  levels  is  resistant  to  forgetting  (Craik  & 
Lockhart,  1972).  These  elements  of  drill-and-practice,  active  learning,  and 
dual-modality  processing  often  found  in  CBI  programs  would  then  seem  to  fa¬ 
cilitate  a  more  permanent  transfer  of  information  than  would  be  offered  by 
other  instructional  media. 

Clark  and  his  associates  (Clark  &  Salomon,  1985  and  Clark  A  Voogel,  1985) 
challenged  the  assumption  that  CBI  would  augment  students'  transfer  abili¬ 
ties.  Clark  and  Voogel  have  suggested  that  current  CBI  courseware  is  geared 
toward  helping  students  to  make  procedural  level  transfer  but  not  cognitive- 
level  transfer.  Bialo  and  Erickson  (1984),  after  evaluating  the  courseware 
for  163  microcomputer  programs,  found  that  70?  of  the  courseware  dealt  solely 
with  procedural  level  objectives.  Clark  and  associates  correspondingly  have 
indicated  that  CBI  has  minimal  influence  upon  students'  long-term  transfer 
and  memory  process.  Cognitive  effects  cited  to  be  associated  with  CBI,  e.g., 
deeper  processing  of  information,  are  not  necessarily  unique  to  this  medium 
(Clark  &  Salomon). 

Limited  research  evidence  has  been  collected  about  CBI's  potential  effec¬ 
tiveness  for  facilitating  either  cognitive  processes  or  transfer.  Gleason 
(1981);  McDonald  and  Crawford  (1983);  and  Tatsuoka  and  Misselt  (1978)  have 
suggested  that  retention  following  CBI  training  is  substantial.  As  stated  by 
Gleason:  "Retention  following  CAI  is  at  least  as  good  if  not  superior  to 
retention  following  conventional  instruction."  (pl6)  Nevertheless,  Gleason 
does  not  provide  any  data  to  verify  this  claim.  Furthermore,  McDonald  and 
Crawford  failed  to  compare  the  retention  obtained  by  CBI's  subjects  with  a 
control  group's  retention  performance,  and  Tatsuoka  and  Misselt' s  findings 
seem  to  be  an  artifact  of  the  lesson's  being  inadvertently  repeated  because 
of  hardware  problems. 

Several  other  reports  (e.g.,  Swezey,  Criswell,  Huggins,  Hays,  &  Allen,  in 
press;  Swezey,  Huggins,  &  Perez,  in  press;  and  Pagliaro,  1983)  failed  to  find 
any  significant  effects  for  CBI  upon  either  subjects'  transfer  or  retention 
performance .  Pagliaro  found  that  pharmacology  students'  retention  perform¬ 
ance  was  the  same  for  both  CBI  and  conventional  instruction  training.  Swezey, 
Criswell  et  al.  found  that  practice  on  a  3-D  module  component  was  more  in¬ 
strumental  than  was  the  instructional  media  (CAI,  CRT  or  videotape)  in  fa¬ 
cilitating  military  students'  transfer  on  a  simple  engine  repair  procedural 
task.  The  same  results  were  replicated  by  Swezey,  Huggins,  and  Perez  for  a 
complex  procedural  task.  CBI's  potential  for  facilitating  retention  of 
procedural  tasks  must  be  further  examined  because  military  CBI  programs  are 
frequently  used  for  such  tasks. 

Personal  experience  has  indicated  that  incidental  learning  is  an  impor¬ 
tant  part  of  conventional  instruction.  Students,  for  instance,  in  an  educa¬ 
tional  psychology  class  may  incidentally  learn  from  the  teacher's  mannerisms 


and  class  discussions  teaching  techniques  which  may  last  a  lifetime.  Inci¬ 
dental  learning  is  another  aspect  of  transfer  which  has  rarely  been  examined. 
Such  incidental  learning  may  indeed  not  be  possible  with  CBI  because  of  pro¬ 
gramming  limitations.  In  this  case  the  spontaneity  and  richness  found  in 
classroom  environments  may  not  be  completely  duplicated.  Limitations  in  this 
medium  may  then  be  limiting  the  courseware  and  not  the  courseware  limiting 
the  medium  as  previously  indicated.  However,  some  important  incidental 
learning  may  be  occurring  in  CBI  which  evaluators  are  not  measuring.  For 
instance,  a  CBI  military  training  course  on  maintenance  may  also  help  the 
students  to  become  better  readers .  Even  though  measuring  incidental  learning 
is  difficult,  information  is  needed  about  CBI's  impact  upon  such  learning.  As 
noted  by  Reeves  and  Lent  (1982),  "The  sponsors  and  consumers  of  CBI  should  be 
informed  as  much  as  possible  about  the  ultimate  effects  of  this  medium  when 
they  make  decisions  about  continuance,  expansion,  or  selection."  ( p 1 3 ) 

Human  Factors  and  Effectiveness 

Sawyer  (1985)  noted  the  importance  for  examining  human  factors  issues  in 
CBI  evaluations.  According  to  Sawyer,  evaluating  human  factors  issues  in¬ 
volves  examining  the  effects  of  a  system’s  hardware  and  documentation  upon 
students'  performance. 

Kearsley  et  al.  (1983)  have  claimed  that  a  CBI  system's  response  time  and 
downtime  are  important  factors  in  the  success  of  a  CBI  project.  These  dimen¬ 
sions  may  affect  user's  mastery  of  the  instructional  information.  Tatsuoka 
and  Misselt  (1978)  noted  that  mechanical  failures  with  the  PLATO  system  had 
negatively  effected  US  Air  Force  students'  learning  efficiency.  Educational 
experts  not  in  the  CBI  field  have  stressed  the  importance  of  measuring  the 
relationship  between  educational  productivity  and  a  medium's  downtime.  Gump 
(1985)  suggested,  for  example,  that  a  downtime  of  1 0%  in  a  conventional  in¬ 
struction  program  could  equal  an  instructional  efficiency  loss  of  1 0%.  Com¬ 
parisons  between  CBI  and  other  instructional  media  should  examine  potential 
instructional  loss  due  to  "downtime"  problems. 

The  civilian  computer  literature  (e.g.,  Chapanis,  1985;  Card,  English,  & 
Burr,  1978;  Phillips  et  al.,  1984;  and  Reigeluth  &  Garfield,  1984)  has  dis¬ 
cussed  the  impact  that  certain  computer  hardware  features  have  upon  users' 
performance.  Card  et  al.  found  significant  differences  in  subjects'  abili¬ 
ties  to  use  four  common  computer  responding  mechanisms — a  mouse,  a  joystick, 
step  keys,  and  text  keys — with  the  mouse  showing  in  the  best  results  (see 
Card  et  al.  for  a  description  of  these  devices).  Chapanis  noted  that  people 
were  normally  able  to  read  materials  presented  on  a  hard-copy  30%  more  effi¬ 
ciently  than  when  they  were  presented  on  a  CRT.  Phillips  et  al.  reported 
that  problems  with  screen  visibility  on  CRT's  impeded  the  educational  per¬ 
formance  of  secondary  school  students.  They  also  found  that  awkwardly  de¬ 
signed  keyboards  and  terminals  poorly  placed  in  classrooms  led  to  educational 
problems. 

Documentation  problems  may  also  adversely  affect  CBI's  education  poten¬ 
tial.  One  cannot  fully  understand  nor  use  ary  instructional  procedures  with¬ 
out  adequate  source  materials.  Bialo  and  Erickson  (1984)  found  the 
documentation  to  be  barely  adequate  for  most  of  the  163  microcomputer  systems 

18 


reviewed  and  information  about  instructional  materials  was  absent  or  inade¬ 
quate  for  62$  of  these  programs.  Chapanis  (1965;  1985)  also  found  documenta¬ 
tion  materials  for  computers  to  be  inadequate.  He  has  shown  that  reading  a 
system's  documentation  requires  the  reader  to  be  familiar  not  only  with  nor¬ 
mal  computer  jargon  but  also  the  particular  system's  jargon. 

Conclusions 

Inconsistent  findings  have  been  obtained  about  CBI's  potential  impact 
upon  students'  learning,  and  positive  instructional  effects  found  for  CBI 
might  be  traceable  to  factors  other  than  the  medium.  Most  pronounced  among 
these  non-medium  effects  were  practice  differences  and  instructional  content 
differences  between  CBI  treatment  and  that  of  the  comparative  medium.  As¬ 
sumptions  about  computers  making  instruction  more  effective  have  thus  not 
been  validated.  Human  factors  and  courseware  problems,  however,  may  limit 
the  computer's  potential  effectiveness,  especially  for  facilitating  cognitive 
skills.  A  basic  unresolved  question  in  CBI  research  is  whether  possible 
problems  with  CBI  are  due  to  such  courseware  and  human  factors  limitations  or 
due  to  the  limitations  of  the  medium. 

CBI  researchers  need  to  obtain  more  conclusive  data  on  the  long-term 
effects  of  CBI.  A  primary  function  of  any  educational  program  is  to  insure 
that  long-term  changes  occur  within  the  students.  The  military  is  especially 
interested  in  troops  retaining  skills  and  knowledge  for  an  indefinite  period 
of  time. 
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CBI  and  Individual  Differences 


Handling  individual  learning  differences  has  long  been  a  difficult  prob¬ 
lem  for  educators.  Many  CBI  professionals  (e.g.,  Avner,  et  al.,  1980;  Car¬ 
rier,  1979;  Kearsley  et  al.,  1983;  Merrill  et  al.,  1980;  and  Steinberg,  1 98H ) 
believe  that  this  medium  is  ideally  suited  to  handle  individual  learning  dif¬ 
ferences.  As  stated  by  Carrier,  "To  many  educators,  the  computer  represents 
the  ultimate  individual  difference  machine. "(p25) 

CBI's  superiority  over  other  media  would  then  be  most  pronounced  for 
meeting  the  educational  needs  of  all  students.  Arguments  supporting  this 
position  have  come  from  CBI  reports  on  learner  control  mechanisms  and  cogni¬ 
tive  differences.  Reports  by  Carroll  (1984);  Hess  &  Miura  (1985);  and  Wood  & 
Pitz  (1985)  have  suggested,  however,  that  some  students  may  have  problems 
learning  from  a  CBI  program. 

Learner  Control  Mechanisms 


A  basic  belief  held  by  many  educators  is  that  students  should  be  encour¬ 
aged  to  make  educational  choices  for  themselves  (Carrier,  Davidson,  Higgson, 

&  Williams,  1984).  Such  choices  would  enable  students  to  more  actively  proc¬ 
ess  the  instructional  material.  As  previously  indicated,  such  active  learn¬ 
ing  would  lead  to  superior  learning.  Learner  control  is  also  expected  to 
help  educators  deal  with  individual  learning  differences.  Gay  (1985)  has 
noted  that: 

It  is  commonly  assumed  that  individual  differences  in  abilities 
and  aptitudes  will  be  accommodated  if  learners  have  more  control 
over  the  pace,  amount  of  practice,  or  style  of  instructions  they 
receive .(pi ) 

CBI  designers  have  developed  learner  control  mechanisms  which  provide 
students  with  some  degree  of  freedom  in  selecting  the  instructional  materials 
to  study.  TICCIT  students,  for  example,  can  select,  through  a  special  key¬ 
board,  the  display  best  suited  for  their  educational  needs  (Merrill  et  al, 
1980).  Merrill  et  al.  have  also  claimed  that  this  learner  control  mechanism 
differentiates  TICCIT  from  other  instructional  media,  including  PLATO  and 
some  other  CBI  systems. 

Some  research  evidence  (e.g..  Collier,  Foynor,  O'Neill,  A  Judd,  1973; 
Fishman,  1985;  and  Hansen,  1972)  exists  supporting  the  use  of  learner  control 
mechanisms  over  program  control  (computer  controlled  sequences) .  Hansen 
found  that  learner  control  subjects  exhibited  greater  decreases  in  state 
anxiety  than  did  control  subjects.  Collier  et  al.  also  found  reduced  anxiety 
levels  to  be  associated  with  using  learner  control  mechanisms.  Increased 
learning  usually  occurs  when  anxiety  toward  the  instructional  material  is 
reduced .  Fishman  observed  that  learner  control  mechanisms  allowed  students 
to  have  the  additional  practice  needed  to  most  effectively  master  the  mate¬ 
rial  .  She  found  that  learner  control  students  outperformed  students  who 


received  instruction  from  lecture  and  video  presentations.  Perhaps  then, 
learner  control  mechanisms  can  make  CBI  a  unique  and  superior  instructional 
medium. 


Several  reports  (e.g..  Ford,  Slough,  &  Hurlock,  1972;  Qay,  1985;  Judd, 
Daubek,  4  O’Neil,  1975;  Snow,  1980;  Steinberg,  1977;  and  Tennyson  &  Buttrey, 
1980),  however,  have  challenged  the  previously  cited  research  about  the  in¬ 
structional  effectiveness  of  learner  control  features.  Steinberg  in  a  com¬ 
prehensive  review  of  this  literature  concluded  that  students  who  were  given 
complete  control  over  course  flow  took  longer  to  complete  the  course  than  did 
control  students.  Also,  experimental  and  control  students  scored  equally 
well  on  achievement  tests.  Ford  et  al.  did  not  find  any  improvements  in  Navy 
students'  learning  in  the  Basic  Electricity  and  Electronics  School  associated 
with  using  learner  control  mechanisms.  Tennyson  &  Buttrey  noted  that  when 
students  controlled  the  instructional  process,  they  often  terminated  the 
process  too  early  and  did  not  effectively  learn  the  materials.  Judd  et  al. 
furthermore,  concluded  that  a  complex  interaction  existed  between  subjects* 
ability  to  use  learner  control  mechanisms  and  the  subjects'  characteristics. 
Students  with  high  Achievement- Independence  scores--as  measured  by  the  Cali¬ 
fornia  Psychological  Inventory — were  the  students  most  able  to  use  the  learn¬ 
er  control  features. 

Gay  (1985)  found  that  learner  control  mechanisms  did  not  accommodate 
individual  background  differences.  Subjects  with  high  prior  conceptual  un¬ 
derstanding  of  the  material  made  significantly  better  use  of  these  control 
options  than  did  subjects  with  poor  prior  understanding.  Reinking  (1983)  and 
Carrier  et  al.  (1984)  also  found  that  control  options  did  not  accommodate 
individual  learning  differences.  Reinking  found  that  the  use  of  learner 
control  options  did  not  improve  poor  readers'  level  of  reading.  All  students 
may  then  not  be  able  to  use  and  benefit  from  learner  control  options. 


Cognitive  Differences 

If  CBI's  instructional  impact  was  not  found  for  the  learner  control  is¬ 
sue,  it  might  then  be  discovered  by  examining  the  cognitive  differences  lit¬ 
erature.  Many  reports  (Allen  &  Merrill,  1985;  Bangert-Drowns  et  al.,  1985; 
Farr,  1985;  Gray  et  al.,  1985;  Neimiec  &  Walberg,  1985;  and  Zemke,  1984)  have 
suggested  that  the  main  reason  for  selecting  a  CBI  system  is  to  better  deal 
with  differences  in  cognitive  abilities  and  styles. 

Most  studies  concerning  disadvantaged  learners  have  used  highly  struc¬ 
tured  drill-and-practice  programs  (see  such  projects  as  the  Cleveland  Public 
Schools,  1981  and  Mevarech  &  Rich,  1985).  Such  courseware  seemed  to  provide 
poorer  learners  with  the  needed  additional  structure  of  and  practice  with  the 
materials.  As  Bangert-Drowns  et  al.  discovered,  CBI  had  the  most  effect  upon 
increasing  low  ability  students'  test  scores  than  for  any  other  usage.  The 
meta-analysis  comparison  between  CBI  and  conventional  instruction  revealed  an 
effect  size  of  .46  in  favor  of  CBI  for  teaching  low  ability  subjects.  Zemke 
(1984)  also  observed  that  low  achievers  seemed  to  gain  a  great  deal  from  CBI. 


Problems  do  exist  with  selecting  a  CBI  program  based  on  this  literature. 
Mevarech  &  Rich  (1985),  Reiss,  Bass,  and  Sharpe  (1986);  and  Zemke  (1984) 
noted  that  CBI  programs  were  the  most  effective  when  used  to  supplement  the 
classroom  instruction.  As  stated  in  the  previous  section,  many  of  these  sup¬ 
plemental  studies  did  not  include  a  comparative  control  group.  Improvements 
in  disadvantaged  students'  learning  could  have  then  been  produced  by  using 
any  instructional  medium  to  supplement  the  conventional  instruction  program. 

Drill-and-practice  programs  might  not  be  useful  for  all  subjects.  Such 
courseware  might  be  too  structured  and  limited  for  most  students,  especially 
gifted  students.  B.  Clark  (1983)  has  concluded  that  gifted  students  need 
tasks  which  allow  them  the  freedom  to  proceed  beyond  a  rudimentary  rehashing 
of  the  materials.  Since  drill-and-practice  programs  dominate  the  market, 
most  current  CBI  programs  may  have  limited  value  for  gifted  students. 

Data  from  a  study  conducted  by  Barsam  &  Simutis  (1984)  provided  some 
insight  into  courseware  which  might  be  geared  toward  certain  kinds  of  gifted 
students.  This  study  explored  the  relationships  between  individual  differ¬ 
ences  in  spatial  abilities  and  soldiers'  performance  on  a  CBI  terrain  visu¬ 
alization  task  consisting  of  three-dimensional  graphics.  This  study  also 
examined  differences  in  soldiers'  abilities  to  benefit  from  courseware  which 
utilized  either  active  (subjects  select  the  graphic  for  viewing)  vs.  passive 
(computer  selects  the  graphic  for  the  subjects'  viewing)  training  procedures. 
They  found  that  the  high  spatial  ability  subjects  were  better  able  to  com¬ 
plete  the  lesson  successfully  and  benefit  from  the  active  training  procedures 
than  were  soldiers  with  normal  and  low  spatial  abilities. 

The  high-spatial  ability  students  seemed  to  have  learner  characteristics 
similar  to  "field-independent"  students  (Witkin,  Goodenough,  &  Karp,  1967), 
who  are  usually  the  high  achievers  in  math  and  science  courses.  (See  Witkin, 
Moore,  Goodenough,  &  Cox,  1977  for  a  further  description  of  the  relationship 
between  field-independence  and  math/science  aptitudes) .  Field-independent 
students  are  also  quite  analytical  in  viewing  three-dimensional  objects  and 
can  also  benefit  from  self-structuring  the  instructional  environment.  Corre¬ 
spondingly,  then,  students  with  mathematical  and  scientific  aptitudes  would 
be  the  mo3t  likely  to  successfully  handle  CBI  lessons,  e.g.,  simulation  pro¬ 
grams  with  numerous  complex  graphics.  These  last  few  paragraphs  suggest 
that:  (a)  the  basic  drill-and-practice  CBI  programs  favor  the  poorer 

learners  while  (b)  the  advanced  CBI  simulation  courseware  is  geared  toward 
the  more  "gifted  students,"  especially  in  mathematics  and  science. 

Variability  in  students'  abilities  to  use  different  types  of  CBI  programs 
should  then  relate  to  cognitive  style  differences.  Witkin  et  al.  (1977) 
claimed  that  individual  differences  along  the  field  independence/dependence 
continuum  reflected  basic  differences  in  people’s  characteristic  styles  of 
dealing  with  the  environment.  This  continuum  does  not  differentiate  students 
with  different  I.Q.  levels.  It  does  differentiate  students  with  different 
modes  of  handling  instructional  materials.  Field-dependent  people  rely  on 
external  sources,  e.g.,  teachers,  to  structure  and  help  understand  the  com¬ 
plex  perceptual  and  abstract  stimuli,  e.g.,  computer  graphics,  while  field- 
independent  people  do  not  need  such  external  aids.  And,  as  previously  indi¬ 
cated,  field-independent  people  are  better  able  to  more  effectively  use  the 


information  inherent  in  such  situations.  Witkin  et  al.  have  also  observed 
that  instructional  situations  with  explicit  instructions,  concrete  presenta¬ 
tion  of  stimuli,  and  explicit  information  about  performance  outcomes  favor 
field-dependent  students.  Field-independent  students  are  better  able  to 
perform  on  more  abstract  and  less  structured  tasks.  Che  would  then  expect 
that  drill-and-practice  CBI  systems  with  structured  instructional  sequencing 
and  continuous  performance  feedback  would  be  the  most  appropriate  for  field- 
dependent  people.  CBI  programs  with  complex  graphics  and  learner  control 
features — as  described  in  the  previous  paragraphs — would  be  more  geared  to¬ 
ward  field-independent  people. 

Limited  and  contradictory  evidence  exists  about  the  actual  relationship 
between  current  CBI  programs  and  cognitive  styles.  Carrier  et  al.  (1984)  and 
Park  and  Roberts  (1985)  found  data  which  indicated  differences — as  expected — 
in  CBI  performance  associated  with  cognitive  style  differences.  Park  and 
Roberts  found  that  field-dependent  people  did  need  more  guided  instruction  to 
complete  a  set  of  CBI  lessons  than  did  field-independent  people.  Carrier  et 
al,  however,  did  not  find  differences  between  field-independent  and  field- 
dependent  subjects  in  their  use  of  control  options.  Burger  (1985)  did  not 
find  any  significant  relationships  between  students’  cognitive  styles  and  CBI 
preference  and  performance. 

An  instructional  system’s  adaptability  to  a  students’  cognitive  style  is 
a  major  issue  addressed  by  Witkin  et  al.  Indeed,  they  emphasized  the  need 
for  the  educational  medium  to  match  the  learners'  cognitive  style.  Several 
CBI  advocates  (e.g.,  Kearsley,  et  al.,  1983;  J.  Olsem,  personal  communica¬ 
tion,  October,  1985;  and  Tennyson,  Christiansen,  and  Park,  1984)  have  sug¬ 
gested  that  the  new  generations  of  CBI  systems  with  intelligent  programs 
(smart  technology)  can  make  the  instructional  system  adaptable  to  students' 
learning  styles.  Olsem  has  claimed  that  these  ICBIs  (Intelligent  Computer- 
based  Instruction)  have  the  capacity  to  modify  the  instructional  mode  to  best 
meet  the  students'  learning  profiles.  These  learning  profiles  are  determined 
by  cognitive  tests  embedded  in  the  first  phase  of  the  ICBI’s  instructional 
program . 

Staart  technology  can  also  be  used  for  accommodating  students  with  differ¬ 
ent  cognitive  abilities  (Allen  4  Merrill,  1985;  and  Gray  et  al.,  1985).  Smart 
technology  programs  involve  interactions  between  students  and  the  computer  so 
that  the  type  of  feedback  and  instructional  materials  provided  fit  the  stu¬ 
dents’  level  of  knowledge  (Gray  et  al.).  They  have  also  claimed  that  the 
promise  of  this  technology  is  that  each  student's  unique  cognitive  needs  will 
be  met.  Unfortunately,  these  smart  technology  programs  have  neither  been 
fully  implemented  nor  tested. 

Adaptive  control  programs- -a  form  of  smart  technology — have  been  shown  to 
be  effective  for  facilitating  students'  learning  (Avner  et  al.,  1980;  Ross, 
1984;  and  Tennyson  4  Buttrey,  1980).  An  adaptive  control  is  when  the  amount 
of  instruction  is  managed  by  the  CAI  program  based  upon  students'  achieve¬ 
ment-level,  aptitude,  or  cognitive-style  (Carrier  et  al,  1984)  Ross  has 
shown  that  such  a  program  can  accommodate  individual  learning  differences. 
Furthermore,  Avner  et  al.  have  claimed  that  adaptive  control  programs  repre¬ 
sent  the  real  basis  for  CBI's  superiority. 


CNI  is  another  adaptive  control  mechanism  which  is  seemingly  suitable  for 
accommodating  individual  differences  (Baker,  1978;  Federico,  1983;  and 
Kearsley  et  al.,  1983)*  CMI  can  be  used  to  advise  students  and  teachers 
about  the  most  appropriate  instructional  sequence  for  each  student.  Federico 
showed  that  for  Navy  trainees  CMI  programs  reduced  the  learning  variability 
usually  associated  with  cognitive  differences. 

The  supportive  research  for  adaptive  control  mechanisms  may  be  problem¬ 
atic.  Research  on  smart  technology  applications  have  been  experiments  of 
short  duration  which  have  rarely  compared  CBI  to  other  media.  These  studies 
have  also  mainly  investigated  CBI  programs  in  laboratory-type  settings.  Ques¬ 
tions  thus  remain  about  the  relative  effectiveness  of  smart  technology  pro¬ 
grams  for  educational  settings.  And,  Ross  (1984)  has  demonstrated  only 
limited  advantages  for  CMI  over  other  media  for  dealing  with  learning  differ¬ 
ences.  CMI  was  only  found  to  be  advantageous  for  complicated  and  lengthy 
lessons. 

Problems  may  also  exist  with  using  adaptive  CBI  programs.  CMI  programs 
are  used  in  conjunction  with  either  conventional  instruction  or  CAI  programs. 
CMI  is  thus  useful  as  a  supplemental  but  not  as  a  primary  mode  of  instruc¬ 
tion.  Also,  Merrill  et  al .  (1980)  felt  that  adaptive  CAI  programs  would 
ultimately  be  "maladaptive"  for  meeting  students'  educational  needs  because 
students'  subsequent  learning  in  non-CAI  environments  would  become  more  dif¬ 
ficult  as  the  real  world  does  not  adapt  to  the  student. 


Individual  Differences  and  CBI  Limitations 

As  discussed  previously,  CBI  may  not  be  beneficial  to  all  students.  Sev¬ 
eral  investigators  (e.g.,  Cambre  &  Cook,  1985;  Jay,  1981;  Loyd  &  Gressard 
1984a  &  1984b;  Maurer,  1983;  and  Wood  and  Pitz,  1985)  have  found  that  com¬ 
puter  anxiety  can  inhibit  a  person's  ability  to  successfully  master  computer 
skills;  consequently,  such  anxiety  may  inhibit  a  student's  mastery  of  infor¬ 
mation  from  a  CBI  system.  As  discussed  by  Cambre  &  Cook,  the  following  four 
behaviors  are  associated  with  computer  anxiety  (p.52): 

(1)  Avoidance  of  the  computer  and  general  area  where  the  computer  is 
located . 

(2)  Excessive  caution  with  the  computer. 

(3)  Negative  remarks  about  the  computer. 

(4)  Attempts  to  cut  the  computer  session  short. 

The  existence  of  and  problems  associated  with  computer  anxiety  have  not 
been  systematically  investigated.  Loyd  and  Gressard  (1984b)  have  attempted 
to  rectify  this  problem  in  the  CBI  literature  by  devising  a  self-report  ques¬ 
tionnaire  which  measures,  among  other  things,  these  cited  dimensions  of  com¬ 
puter  anxiety.  They  have  claimed  that  this  questionnaire  provides  an  effec¬ 
tive,  reliable,  and  convenient  measure  of  students'  computer  anxiety.  Wood 


and  Pitz  (1985),  however,  have  suggested  that  any  self-report  questionnaire 
on  computer  anxiety  (e.g.,  Loyd  and  Gressard' s,)  without  scales  for  lying  and 
social  desirability  may  not  be  a  valid  measure,  because  subjects'  stated 
expectations  about  using  a  computer  are  not  always  congruent  with  their  ac¬ 
tual  computer  behaviors.  Personal  experiences  have  also  demonstrated  that 
people  may  be  reluctant  to  state  their  fears  of  using  a  computer. 

A  reliable  and  valid  questionnaire  on  computer  anxiety  would  be  benefi¬ 
cial  to  both  educators  and  researchers.  Such  a  questionnaire  would  provide 
additional  insights  into  explaining  possible  variability  in  students'  abili¬ 
ties  to  use  CBI  materials.  This  scale  would  also  then  help  educators  to 
pinpoint  those  students  who  need  additional  support  and  guidance  for  using 
this  medium.  Schubert  (1985),  for  example,  found  that  computer  anxious  fe¬ 
males  had  to  go  through  a  formal  desensitization  process  before  they  were 
able  to  deal  with  the  CBI  materials.  Consequently,  the  educational,  temporal 
and  financial  costs  associated  with  helping  such  computer  anxious  students 
should  also  be  included  in  any  evaluation  of  a  CBI  system. 

Differences  in  computer  experience  may  also  differentially  affect  stu¬ 
dents'  use  of  CBI  materials.  Carroll  and  his  associates  (e.g.,  Carroll, 

1984;  Carroll,  &  Mazur,  1984;  and  Carroll  &  Thomas,  1982)  have  noted  that 
infrequent  computer  users  often  have  difficulty  in  learning  to  use  a  computer 
system.  They  also  suggested  that  embedded  instructions  for  using  a  computer 
must  relate  to  students'  prior  computer  use  and  knowledge.  CBI  professionals 
must  then  consider  both  novice  and  experienced  computer  populations  when 
designing  and  evaluating  a  CBI  system.  If  the  system  cannot  accommodate  all 
levels  of  users,  then  a  segment  of  students,  especially  the  infrequent  users, 
will  not  be  able  to  use  and  benefit  from  the  CBI  system.  There  also  may  be 
extreme  variability  in  subjects'  training  t  ime  on  3  CBI  system,  with  infre¬ 
quent  users  proceeding  much  slower  then  regular  and  frequent  users.  Such 
variability  may  then  negate  CBI's  previously  cited  potential  for  reducing 
instructional  time.  This  variability  may  decrease  in  military  training  as 
the  number  of  military  students  taking  computer  literacy  and  CBI  courses 
increases. 

Computer  experiences  may  also  relate  to  computer  anxiety  (Loyd  & 

Gressard,  1984a;  and  King,  1975).  Even  though  the  data  may  be  problematic, 
Loyd  and  Gressard  did  find  a  significant  relationship  between  infrequent 
computer  use  and  computer  anxiety.  If  this  is  so,  then  computer-experienced 
users  would  likely  be  excited  and  motivated  to  use  a  novel  CBI  program  while 
non-experienced  users  might  be  apprehensive  about  using  such  a  program.  This 
novelty  issue  will  be  further  discussed  in  the  section  on  student  attitudes. 

Hess  and  his  associates  (e.g.,  Hess  &  Miura,  1985;  Hess  &  Tenezakis, 

1973;  Miura  &  Hess,  1983,  and  Schubert,  1985)  have  indicated  that  there  are 
sex  differences  associated  with  students'  use  of  computers.  Miura  &  Hess 
have  found  that  middle-school  boys  more  than  girls:  a)  owned  and  used  home 
computers  and  b)  enrolled  in  computer  camps  and  summer  classes.  They  also 
asked  these  students  to  rate  a  number  of  randomly  selected  courseware  titles 
as  being  either  male  of  female  oriented.  These  students  rated  a  significant¬ 
ly  larger  number  of  courseware  titles  to  be  primarily  suited  for  male 


audiences  and  not  for  female  audiences.  Other  studies,  e.g.,  Loyd  and 
Gressard  (1984a),  however,  did  not  find  gender  differences  to  be  significant¬ 
ly  related  to  computer  use  or  anxiety. 

Conclusions 

CBI's  potential  as  the  "ultimate  individual  difference  machine"  has  not 
been  substantiated.  Even  though  this  potential  may  become  more  evident  for 
future  CBI  generations,  research  on  current  CBI  programs  and  individual  dif¬ 
ferences  is  plagued  with  confounded  and  contradictory  findings. 

Individual  differences'  may  conversely  limit  CBI's  usefulness.  As  dis¬ 
cussed,  computer-anxious  students  are  the  most  likely  to  have  problems  with 
using  this  medium.  Finally,  the  educational  and  financial  costs  associated 
with  helping  students  to  successfully  use  a  CBI  program  must  also  be  included 
in  a  system's  cost-analyses. 


CBI  and  Student  Attitudes 


The  most  seemingly  pervasive  finding  in  the  CBI  literature  is  that  stu¬ 
dents  favor  taking  a  course  by  this  medium.  Many  reports  (e.g.,  Clement, 
1981;  Merrill  et  al.,  1980;  and  Myers,  1 984 )  have  urged  implementation  of  CBI 
based  upon  student  attitudes.  Myers  noted  that  PLATO  has  been  enthusiastic¬ 
ally  accepted  by  students  and  teachers.  Also,  advertisements  on  CBI  products 
have  claimed  that  "learner  motivation  is  high  because  the  system  is  easy  and 
fun  to  use."  (Teaching/Learning  Technologies,  1984) 

As  the  research  on  students’  CBI  attitudinal  data  has  rarely  been  exam¬ 
ined,  the  robustness  of  the  data  about  students’  attitudes  toward  CBI  as 
compared  to  their  attitudes  toward  other  media  must  be  questioned.  This 
analysis  must  also  include  examining  the  pyschometric  attributes  of  the  dif¬ 
ferent  attitudinal  measures  used  in  these  studies.  In  this  section,  factors 
influencing  students'  CBI  views  will  also  be  discussed. 

Attitudinal  Studies 


Comparisons  of  students'  attitudes  toward  CBI  and  other  media  have  been 
included  in  some  of  the  previously  cited  military  studies  (e.g.,  Dallman  et 
al.,  1977;  Ford  et  al.,  1972;  Longo,  1972;  and  Tatsuoka  &  Misselt,  1978). 
These  studies  indicated  that  students  favored  taking  a  course  by  CBI  rather 
than  by  other  media.  Longo,  for  example,  found  in  two  separate  studies  more 
positive  ratings  being  exhibited  by  the  CBI  group  than  by  the  conventional 
instruction  group. 

Dallman  et  al.  (1977)  also  reported  that  students  tended  to  favor  taking 
a  CBI  course  rather  than  a  conventional  instruction  course.  They  noted, 
however,  that  the  conventional  instruction  course  was  taught  by  newly  trained 
teachers.  They  also  found  that  a  conventional  instruction  course  taught  by  a 
popular  teacher  was  more  favorably  received  than  was  CBI.  In  a  second  study, 
they  failed  to  find  any  significant  differences  between  students’  attitudes 
toward  CBI  and  toward  programmed  instructions. 

Mixed  results  about  students'  preference  for  CBI  over  other  instructional 
media  were  the  pattern  for  the  civilian  literature.  Avner  and  associates 
(Avner,  1981;  and  Jones,  Kane,  Sherwood,  &  Avner,  1983)  in  a  series  of  course 
evaluations  found  that  University  of  Illinois'  students  favored  PLATO  over 
other  instructional  media.  For  two  semesters.  Chemistry  100  students  were 
asked  to  rate  several  different  media  with  regards  to  helpfulness  in  learning 
the  class  materials.  These  media  were  PLATO,  textbook,  lecture,  labs,  and 
quiz  sections.  The  main  professor  for  this  course  had  received  several  out¬ 
standing  teaching  honors.  For  both  semesters,  the  PLATO  system  received  the 
highest  ratings  of  4.5  on  a  five  point  scale.  Differences  between  ratings 
for  CBI  and  conventional  instruction  were  substantial — over  a  point  differ¬ 
ence  on  a  five  point  scale.  Differences  in  students'  ratings  of  CBI  and  Quiz 
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section  were  .9  and  .4  for  Fall,  1977  and  Spring,  1978  semesters,  respective¬ 
ly.  These  course  evaluations  and  Dallman  et  al's  study  are  some  of  the  few 
studies  in  which  students'  attitudes  toward  CBI  were  compared  with  their 
attitudes  toward  other  non-conventional  modes  of  instruction. 


Bangert-Drowns  et  al.  (1985)  and  Kulik  et  al.  (1980)  demonstrated  limited 
support  for  the  assumption  that  students  preferred  CBI.  Both  meta-analyses 
could  only  find  a  few  studies--eleven  for  the  Bangert  et  al.  analysis  and 
five  for  the  Kulik  et  al.  analysis — which  compared  attitudes  toward  CBI  and 
toward  conventional  instruction.  Bangert  et  al.  found  an  average  effect  of 
.09  for  CBI  upon  secondary  students'  attitudes,  and  Kulik  et  al.  discovered 
an  effect  of  .18  favoring  CBI  for  college  instruction. 

Problems  with  concluding  that  students  prefer  to  take  a  CBI  rather  than  a 
conventional  instruction  course  are  vividly  demonstrated  by  Alderman  et. 
al.'s  (1978)  findings.  A  large  percentage  of  PLATO  students  indicated  posi¬ 
tive  feelings  toward  this  courseware  as  manifested  by  their  desire  to  take 
another  course  using  this  system.  They  also  disagreed  that  using  PLATO  was 
dehumanizing  or  boring.  Attitudinal  comparisons  between  PLATO  and  conven¬ 
tional  instruction  students,  however,  failed  to  reveal  any  noticeable  differ¬ 
ences  in  these  students'  feelings  toward  their  classes.  Alderman  et  al.  also 
found  that  students  reactions  to  TICCIT  programs  were  usually  less  positive 
than  those  for  the  conventional  instruction  program.  Interestingly  enough, 
this  study  has  been  widely  cited  as  providing  evidence  that  CBI  has  a  posi¬ 
tive  impact  upon  students'  attitudes. 


Magidson  (1978)  is  another  study  widely  cited  as  demonstrating  that  stu¬ 
dents  prefer  to  take  a  CBI  course.  Magidson’s  results  about  PLATO  users' 
views  parallel  those  found  by  Alderman  et  al  (1978).  Over  ninety  percent  of 
these  students  enjoyed  using  PLATO  and  would  recommend  that  their  friends 
take  a  PLATO-based  course.  He  also  reported  that  less  than  ten  percent  of 
these  students  found  PLATO  to  be  boring  and  dehumanizing.  Attitudinal  com¬ 
parisons  between  PLATO  and  conventional  instruction  students  were  not  made. 


Conclusions  cannot  then  be  made  about  students'  preference  for  CBI  over 
other  instructional  media  based  upon  Alderman  et  al's  (1978)  and  Magidson's 
(1978)  studies.  As  indicated,  the  previously  discussed  assumptions  about 
students'  desires  for  CBI  courseware  have  been  based  on  these  two  studies  and 
others  (e.g.,  Dengler,  1983;  and  Jenkins  &  Dangert,  1981)  which  obtained 
parallel  results  using  similar  methodologies.  Nonetheless,  these  studies  do 
suggest  that  students  would  accept  taking  a  course  by  CBI. 


Alderman  et  al's  (1978)  and  Magidson's  (1978)  findings  have  also  suggest¬ 
ed  that  students  do  not  believe  that  CBI  is  dehumanizing  as  an  educational 
experience.  King  (1975)  provided  more  substantial  evidence  for  this  point 
about  CBI  and  the  possible  dehumanization  threat.  After  thoroughly  reviewing 
the  military  and  civilian  CBI  literature  on  student  attitudes.  King  concluded 
that  CBI  is  not  a  dehumanizing  instructional  experience. 


King  (1975)  also  examined  the  data  on  the  relationships  between  student 
attitudes  and  individual  differences.  She  concluded  that  learner  control  has 
not  been  shown  to  significantly  influence  student  attitudes.  However,  some 
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limited  evidence  was  found  showing  that  computers  motivated  underachievers, 
slower  students,  and  potential  dropouts,  Mevarech  and  Rich  (1985)  also  dem¬ 
onstrated  that  CBI  could  improve  disadvantaged  students'  perceptions  of 
school  life.  Again,  these  studies  did  not  systematically  compare  the  rela¬ 
tive  effects  of  CBI  and  conventional  instruction  upon  problem  students'  atti¬ 
tudes. 

Psychometric  Attributes  of  Attitudinal  Measures 


Attitudinal  data  may  be  inconsistent  because  of  problems  with  some  of  the 
instruments  used  in  measuring  these  data.  Knerr  and  Nawrocki  (1978)  have 
noted  that  most  attitudinal  instruments  are  designed  ad  hoc  with  neither  the 
items  not  the  metric  properties  of  the  scales  described.  As  stated  by  King 
(1975): 

Most  (attitudinal)  studies  are  experimental-constructed  tests 

which  have  unknown  or  unreported  reliabilities.  (p7) 

For  example,  Jenkins  and  Dankert  (1981)  indicated  that  reliable  data  were 
collected;  however,  descriptions  of  the  items  and  the  test-retest  reliabili¬ 
ties  were  not  provided.  And  Bangert-Drowns  et  al.  (1985)  and  Kulik  et  al. 
(1980)  failed  to  describe  the  reliabilities  of  the  different  attitudinal 
measures  reviewed  in  their  reports. 

Several  of  the  previously  cited  studies  (e.g.,  Avner,  1981;  Dallman  et 
al.  1977;  Knerr  &  Nawrocki  1978;  Longo,  1972;  Loyd  &  Gressard  1984b  and 
Mevarech  4  Rich,  1985)  have  developed  or  used  attitudinal  measures  with  ap¬ 
parently  sufficient  reliabilities.  Mevarech  and  Rich  measured  students'  per¬ 
ceptions  of  the  instructional  environment  with  a  questionnaire  developed  by 
Epstein  and  McPartland  (1976).  This  questionnaire  had  a  known  internal  con¬ 
sistency  coefficient  of  .84.  Longo  reported  nearly  identical  attitudinal 
scores  on  the  same  items  for  two  separate  studies,  which  indicates  that  the 
measures  were  reliable.  Perhaps,  then,  potential  reliability  problems  with 
attitudinal  measures  can  be  eliminated  by  using  established  questionnaires  or 
by  replicating  the  results.  Other  than  the  few  studies  cited  in  this  report 
(e.g.,  Dallman  et  al’s,  1977  and  Longo's),  a  lack  of  replicated  findings 
comparing  CBI  to  other  media  exists  in  this  research  literature. 

Avner  (  1972)  argued  that  reliability  for  attitudinal  measures  should  not 
be  determined  by  using  a  large  number  of  items  selected  for  internal  consis¬ 
tency.  Rather,  a  highly  reliable  single  item  should  be  used  for  each  attitude 
to  be  measured.  He  has  claimed  that  reducing  the  number  of  items  allows  the 
researcher  to  sample  student  opinions  more  frequently  without  encroaching  on 
the  students'  good  will  or  the  limited  class  time  available.  And  reliability 
measures  which  are  based  upon  internal  consistency  can  be  a  misleading  indi¬ 
cation  of  the  questionnaire's  value,  because  such  a  group  of  items  can  only 
be  vaguely  interpreted.  Except  for  a  few  studies  conducted  by  Avner  and 
associates,  the  reliability  of  single  item  attitudinal  measures  has  rarely 
been  determined. 
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Problems  may  also  exist  with  the  validity  of  the  attitudinal  measures 
used  in  CBI  research.  Discrepancies  between  students'  responses  to  question¬ 
naires  and  more  objective  data  have  been  found  by  Knerr  &  Nawrocki  (1978); 
Shlechter  (1985);  and  Wood  and  Pitz  (1985).  Shlechter,  for  example,  noticed 
that  soldiers  reported  that  using  a  light  pen  was  relatively  easy,  when  in 
fact  many  of  their  errors  related  to  problems  with  using  this  responding 
mechanism.  Additional  efforts  to  compare  self-reports  with  objective  per¬ 
formance  measures  seem  warranted.  As  stated  by  Avner  (1972): 
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While  the  questionnaire  provides  a  convenient  and  useful  method 
of  gathering  student  data  in  certain  situations  (e.g.,  students' 
learning  to  use  a  system)  there  is  no  alternative  to  direct 
observation  of  student  behavior  (p5). 

Bessemer  (personal  communication,  October  31,  1985)  has  suggested 
that  attitudinal  data  must  be  validated  by  students'  actual  use  of  an  imple¬ 
mented  system. 

Another  threat  to  CBI  attitudinal  measures  is  that  items  regarding  social 
desirability  are  not  included.  This  problem  is  especially  important  for 
studies  (e.g.,  Dallman  et  al.  1977;  and  Tatsuoka  and  Misselt,  1978)  in  which 
instructors  have  been  part  of  the  evaluation  process.  Reasons  for  including 
a  social  desirability  measure  will  be  elaborated  upon  in  the  next  few  pages. 

Problematic  attitudinal  measures  are  frequently  found  in  the  CBI  litera¬ 
ture.  As  indicated,  psychometric  problems  with  these  measures  were  observed 
in  studies  with  favorable  and  in  studies  with  neutral  outcomes  about 
student's  preference  for  CBI.  These  measurement  problems  tend  to  further 
limit  conclusions  about  CBI  and  student  attitudes. 


Factors  Affecting  Student  Attitudes 

The  main  factor  affecting  students'  attitudes  should  be  the  CBI  program's 
instructional  value.  Gleason  (1981)  has  claimed  that  students  do  react  very 
positively  to  good  CBI  programs  and  reject  poor  programs.  Clement  (1981)  has 
also  noted  that  CBI  lesson  materials  which  reduce  student  error  rates  may  be 
beneficial  to  maintaining  and  producing  positive  attitudes.  Clement  and 
Gleason  however,  do  not  provide  any  data  supporting  their  positions. 

Limited  evidence  exists  about  the  effects  of  performance  upon  student 
attitudes.  King  (1975)  reported  only  one  study  (Mitzel  &  Wodtke,  1965)  which 
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examined  the  relationship  between  attitudes  and  performance.  Mitzel  &  Wodtke 
found  that  a  significant  positive  correlation  existed  between  these  varia¬ 
bles.  On  the  other  hand,  Tatsuoka  and  Misselt  (1978)  could  not  find  any 
substantial  relationships  between  military  students'  gain-scores  on  a  CBI 
training  program  and  attitudinal  measures.  This  study  investigated  students' 
CBI  use  and  their  attitudes  toward  four  different  courses  at  Chanute  Air 
Force  Base.  Correlations  between  students'  gain-scores  and  attitudinal  in¬ 
dexes  of  instructional  effectiveness  and  positive  affect  were  not  significant 
at  levels  of  .19  and  .21,  respectively.  Significant  correlations  (p  <  .05) 
of  .32  and  -.29  for  the  instructional  effectiveness  and  positive  affect 
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variables,  respectively,  were  found  when  a  multiple  regression  technique  was 
used  to  determine  the  correlational  coefficients.  The  .19  and  .32  correla¬ 
tions  dealt  with  students'  perceptions  of  PLATO'S  effectiveness  and  perform¬ 
ance  while  the  other  two  correlations  involved  students'  positive  feelings 
toward  CBI  and  performance.  Tatsuoka  and  Misselt  never  explained  the  reason 
for  the  directional  shift  in  the  second  set  of  correlations.  Also,  as  re¬ 
ported,  Alderman  et  al.  (1978)  found  that  students  had  less  favorable  atti¬ 
tudes  toward  TICCIT  than  toward  conventional  instruction,  but  TICCIT  was 
found  to  be  more  effective  for  some  educational  measures.  A  link  then  has 
not  been  found  between  attitudes  and  instructional  quality. 

A  link  does  seem  to  exist  between  users'  attitudes  and  human  factors 
issue-  (Avner,  1986,  King,  1975;  Magidson,  1978;  McVey,  Clauer,  &  Taylor, 
1984;  Shlechter,  1985;  and  Tatsuoka  4  Misselt,  1978).  Avner  found  correla¬ 
tions  of  .80  between  users'  (28  instructors)  ratings  and  system  reliability. 
He  also  found  a  significant  point-biseral  correlation  of  .593  between  in¬ 
structors'  ratings  and  the  decision  to  continue/discontinue  the  use  of  the 
PLATO  system.  Tatsuoka  and  Misselt  found  correlations  of  .-43  and  -.45  be¬ 
tween  students'  perceptions  of  mechanical  failures  and  students'  attitudes 
toward  the  system's  acceptability  and  instructional  effectiveness,  respec¬ 
tively.  McVey  et  al.,  discussed  the  possibility  that  hardware  features  might 
affect  users'  comfort.  As  stated  by  McVey  et  al.: 

A  thorough  review  paper  by  Campbell  and  Durden  (1983)  suggests 
that  there  are  many  overlooked  factors  contributing  to  the  hu¬ 
morous  accusation  that  VDT  (CRT)  means  "Visual  Discomfort  Termi¬ 
nals."  (pi) 

Other  reports  from  the  human  factors  literature  (e.g.,  Dainoff,  Happ,  4 
Crane,  1981;  IBM,  1984;  Helander  4  Rupp,  1984;  and  Rupp,  1981)  have  also 
discussed  problems  with  CRTs.  Dainoff  et  al.  found  that  visual  fatique  was 
more  pronounced  among  typists  using  CRTs  than  for  typists  using  print-based 
terminals.  They  also  noted  that  these  differences  in  visual  fatique  were  not 
related  to  amount  of  work  or  other  job  pressures. 

Different  CRT  standards  have  been  recently  developed  by  different  coun¬ 
tries  and  agencies,  e.g..  Department  of  Defense  (DoD)  to  control  for  possible 
problems  with  the  design  of  these  terminals  (Helander  4  Rupp,  1981).  Unfor¬ 
tunately,  as  noted  by  Helander  and  Rupp,  the  DoD  standards  have  emphasized 
productivity  over  operator  comfort  and  convenience.  Possible  instructional 
inefficiency,  however,  may  be  associated  with  "visual  discomfort  terminals" 
as  students  may  not  be  able  to  use  these  terminals  for  an  extended  period  of 
time.  Reliable  attitudinal  data  are  needed  for  ascertaining  specific  problems 
that  students  may  have  with  the  CBI  system. 

Useful  information  about  CBI  programs  has  been  obtained  when  attitudinal 
measures  have  been  used  to  ascertain  specific  issues.  Tatsuoka  and  Misselt 
(1978)  found  that  students’  responses  to  open-ended  attitudinal  questions 
were  helpful  in  revising  poorly  developed  CBI  lessons.  Avner  (personal  com¬ 
munication,  November  1985)  discovered  through  such  open-ended  responses  posi¬ 
tive  attributes  of  CBI,  such  as  scheduling  flexibility  for  students,  which 
would  not  have  been  discovered  through  other  procedures.  And  Shlechter 
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(1985)  found  through  students'  interview  responses  that  a  particular  system 
might  be  best  used  as  a  supplement  to  the  conventional  instructional  program. 
Attitudinal  measures,  especially  open-ended  questions,  may  then  be  most  use¬ 
ful  for  providing  formative  feedback  about  students'  perceptions  of  a  sys¬ 
tem’s  strengths  and  weaknesses. 

Factors  other  than  the  CBI  system  may  unduly  influence  students'  atti¬ 
tudes.  Abrami  and  Mizener  (1982)  noted  that  teacher  attitudes  had  a  substan¬ 
tial  effect  upon  students'  attitudes  in  conventional  instruction  courses. 
Avner  (1985)  demonstrated  that  teachers'  attitudes  toward  CBI  did  effect 
students'  attitudes  toward  the  system.  Avner  used  exemplary  and  mediocre  CBI 
courseware,  as  judged  by  a  panel  of  CBI  professionals.  He  found  that  stu¬ 
dents  initially  judged  both  programs  to  be  either  positive  or  negative  de¬ 
pending  upon  the  teachers'  positive  or  negative  feelings  toward  the  system. 

An  examination  of  the  data,  however,  indicated  that  students  who  had  teachers 
with  positive  attitudes  toward  CBI  still— after  a  month's  period — maintained 
positive  attitudes  toward  the  mediocre  program.  Also,  these  students  had 
slightly  more  positive  attitudes  toward  CBI  than  the  students  in  the 
exemplary  CBI  materials-hostile  teacher  condition.  Teachers' attitudes  could 
then  confound  students'  CBI  attitudinal  data. 

Because  of  the  unique  relationship  between  military  instructors  and  their 
students,  evaluations  of  military  CBI  programs  would  be  especially  prone  to 
problems  associated  with  instructor  attitudes.  Military  instructors  need  to 
be  involved  in  the  evaluation  process.  Draxl  and  Aggen  (1981),  and  Gray 
(1986)  have  described  the  need  to  give  briefings  to  enlist  military  instruc¬ 
tors  interest  in,  and  support  of,  new  instructional  systems.  Military  in¬ 
structors'  attitudes--as  civilian  instructors'  attitudes — may  also  be 
influenced  by  their  superiors'  beliefs.  Indeed  it  is  hard  to  imagine  how 
these  instructors  can  remain  neutral  toward  CBI  when  their  superiors,  e.g., 
Sullivan  (1985),  have  indicated  that  this  medium  will  be  an  integral  part  of 
all  future  Army  training.  Such  briefings  by  CBI  evaluators  and  Army  3taff 
may  engender  positive  attitudes  in  instructors,  while  militating  against 
obtaining  unbiased  reports  from  students.  Military  students  may  be  more 
reluctant  than  civilian  students  to  state  their  dislike  of  a  CBI  system, 
because  military  instructors  have  much  more  power  over  their  students  than  do 
conventional  teachers.  This  possible  difference  between  military  and  civil¬ 
ian  students  can  explain  why  more  favorable  comments  toward  CBI  have  been 
found  in  the  military  training  literature  than  in  the  civilian  literature. 

Another  possible  confounding  in  attitudinal  research  is  novelty  effects. 
As  previously  reported,  Clark  (1983  and  1985a)  has  claimed  that  students  are 
initially  motivated  and  enthusiastic  to  learn  from  a  new  CBI  program.  Ques¬ 
tions,  though,  remain  about  a  system's  ability  to  sustain  students'  interest 
and  motivation.  Pagliaro  (1983)*  in  a  year-long  study  of  a  CBI  program  for 
pharmacology  students,  found  that  these  students  reported  the  system  to  be 
significantly  less  interesting  across  testing  periods.  The  magnitude  of  the 
decrements  in  students'  ratings,  however,  was  not  great. 

Other  CBI  reports  (e.g.,  Avner,  1985;  Ford  et  al.  1972;  King,  1975;  and 
Magidson,  1978)  have  not  been  able  to  find  any  substantial  effects  for  CBI 
experience  upon  attitudinal  data.  Ford  et  al.  found  that  long-term  CBI  users' 


attitudes  toward  CBI  were  more  positive  than  the  attitudes  of  first-time 
users.  The  long-time  users  were  fifty  students  out  of  more  than  200  sampled 
who  had  prior  experience  with  CBI.  Avner  showed  that  subjects  across  the 
different  experimental  conditions  reported  more  positive  attitudes  toward  the 
CBI  system  with  increased  usage.  Thus,  novelty  effects  as  defined  by  length 
of  CBI  experience  do  not  seem  to  have  a  major  influence  upon  students'  atti¬ 
tudes  . 

Clark  and  his  associates  (e.g.,  Clark  &  Salomon,  1985)  also  suggested 
that  students  would  prefer  any  newer  instructional  technologies  over  the 
established  ones — "program  novelty  effects."  Support  for  "program  novelty 
effects"  comes  from  Simonson's  (1980)  extensive  review  of  the  CBI  literature. 
Simonson  has  concluded  that  students  seemingly  prefer  to  learn  from  any  new 
instructional  mode,  e.g.,  instructional  T.V.  or  programmed  instruction,  over 
conventional  instruction.  Shlechter  (1985)  reported  that  students  preferred 
a  CBI  system  to  conventional  instruction  because  of  the  individualized  in¬ 
struction  associated  with  this  system.  These  students  would  then  seemingly 
prefer  any  individual  mode  of  instruction  over  conventional  instruction.  As 
has  been  previously  argued,  individualized  instruction  characterizes  nearly 
all  modern  instructional  innovations. 

Juola  (1977)  reported  findings  which  conflicted  with  those  previously 
cited  about  new  media  and  student  attitudes.  Seven  hundred  freshmen  at 
Michigan  State  University  found  instructional  television  and  programmed  in¬ 
struction  to  be  undesirable  instructional  media.  Juola’ s  study,  however,  is 
not  typical  of  this  research  literature.  Future  research  is  needed  in  which 
students'  CBI  attitudes  are  compared  with  attitudes  for  other:  (a)  equiva¬ 
lent  instructional  media  and  (b)  new  instructional  modes. 


Conclusions 


Inconsistent  and  confounding  findings  are  predominate  in  the  CBI  attitud- 
inal  findings.  Furthermore,  these  problematic  findings  seem  to  relate  to 
factors  other  than  courseware  differences.  CBI  systems  should  not  be  imple¬ 
mented  based  primarily  upon  attitudinal  data.  Recommendations  have  also  been 
made  about  using  attitudinal  data  for  formative  feedback,  making  attitudinal 
comparisons  with  other  new  and  equivalent  instructional  technologies,  using 
objective  measures  to  validate  the  subjective  attitudinal  measures,  and  con¬ 
ducting  more  naturalistic  attitudinal  studies.  Such  naturalistic  studies 
would  ultimately  provide  CBI  professionals  with  the  most  useful  data  about 
CBI  and  student  motivation. 
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Summary  and  Recommendations 


Consistent  empirical  support  does  not  exist  for  the  claimed  advantages  of 
CBI  over  other  instructional  media  for:  a)  reducing  training  time;  b)  reduc¬ 
ing  life  cycle  costs;  c)  facilitating  students’  mastery  of  the  instructional 
materials;  d)  accommodating  individual  learning  differences;  and  e)  motivat¬ 
ing  students'  learning.  The  lack  of  empirical  support  for  these  issues  is 
not  totally  explained  by  problematic  courseware.  As  discussed,  some  course¬ 
ware  was  created  by  some  of  the  finest  instructional  designers.  And  some 
benefits  of  CBI,  e.g.,  training-reductions,  have  been  attributable  to  in¬ 
structional  features  which  are  inherent  in  any  individualized  instructional 
media.  CBI's  inherent  superiority  as  a  primary  instructional  medium  has  thus 
not  been  established. 

Notwithstanding  these  mixed  findings,  future  generations  of  CBI,  e.g., 
ICBI,  do  promise  to  have  a  significantly  positive  impact  upon  students' 
cognitive  processes.  This  medium  also  promises  to  help  slow  learners. 
Unfortunately,  problematic  research  practices  prohibit  any  conclusions  about 
CBI  and  slow  learners.  Also,  CBI  does  not  seem  to  be  a  dehumanizing 
instructional  experience. 

Problematic  research  procedures  were  also  found  throughout  the  CBI  lit¬ 
erature.  Most  noticeable  of  these  research  problems  were:  a)  confoundings 
due  to  differences  in  instructional  content;  b)  making  comparisons  with  inap¬ 
propriate  media;  c)  confoundings  due  to  "program  novelty  effects"  and 
"teacher  attitudes;"  and  d)  findings  which  were  not  replicated.  Recommenda¬ 
tions  have  also  been  made  that  future  CBI  evaluations:  a)  be  conducted  on 
fully  implemented  systems;  b)  examine  hidden  life  cycle  costs  for  each  imple¬ 
mentation;  c)  examine  the  effects  of  human  factors  variables  upon  a  system's 
cost-effectiveness;  d)  examine  the  effects  of  human  factors  variables  upon 
students'  comfort;  e)  examine  students'  abilities  to  transfer  the  instruc¬ 
tional  information;  f)  measure  the  unwarranted  additional  instructions  asso¬ 
ciated  with  the  experimental  or  control  treatments;  g)  control  for  possible 
confoundings  due  to  insufficient  testing  durations;  h)  examine  interactions 
between  students'  characteristics  and  specific  delivery  features  and  CBI 
systems;  and  i)  use  attitudinal  data  for  formative  but  not  summative  evalua¬ 
tion.  Clearer  answers  about  CBI's  inherent  value— in  general  and  for  spe¬ 
cific  system' s--would  be  obtained  if  future  CBI  investigations  deal  with  the 
research  problems  and  recommendations  cited  in  this  paragraph. 

This  paper's  methodological  issues  reflect  the  complexities  in  CBI  evalu¬ 
ation.  Ideally,  CBI  evaluations  should  be  done  by  a  team  of  professionals 
from  various  disciplines--military  or  civilian  education,  human  factors, 
educational  psychology,  and  instructional  technology.  Also  some  time  consum¬ 
ing  and  expensive  evaluation  procedures  must  be  employed.  To  the  extent  that 
CBI  evaluation  is  used  to  compare  CBI  to  other  instructional  media,  especial¬ 
ly  conventional  instruction,  then  both  cross-sectional  and  longitudinal  data 
should  be  collected.  A  cross-sectional  design  is  needed  for  initial  assess¬ 
ments  of  students'  CBI  performance,  while  longitudinal  data  are  necessary  to 


ascertain  the  long-term  learning  and  attitudes  associated  with  the  CBI  sys¬ 
tem.  The  longitudinal  data  can  also  be  used  as  a  reliability  check.  System¬ 
atic  programs  are  needed  in  which  priorities  are  assigned  to  independent 
variables  and  variables  are  systematically  manipulated  and  measured  in  suc¬ 
cessive  studies  using  the  same  CBI  system.  The  systematic  analyses  would 
provide  information  about  the  relationship  between  estimated  and  life  cycle 
costs.  Also  programmatic  research  is  needed  to  provide  further  insights  into 
the  question  of  whether  the  cited  CBI  limitations  are  due  to  courseware  or 
hardware  problems. 

Another  issue  in  CBI  evaluations  concerns  the  traditional  use  of  forma¬ 
tive  evaluations  for  debugging  the  CBI's  programs  problem  areas.  Williges, 
Williges,  and  Elkerton  (in  press)  have  noted  that  formative  evaluation  is 
central  to  the  design  of  any  computer  system.  They  fail,  however,  to  mention 
the  necessity  for  conducting  a  similar  formative  evaluation  upon  the  existing 
or  comparative  instructional  system(s).  These  discrepant  evaluation  proce¬ 
dures  will  certainly  lead  to  the  possibility  of  making  confounded  comparisons 
because  the  CBI  program  is  operating  at  optimal  efficiency  while  the  same 
cannot  be  said  for  the  comparative  media. 

Formative  evaluations  should  also  be  conducted  after  the  initial  experi¬ 
mental  data  have  been  collected,  rather  than  as  traditionally  done  prior  to 
the  experimental  process.  Such  formative  evaluation  data  could  help  deter¬ 
mine  the  reasons  for  the  experimental  findings.  Furthermore,  the  formative 
data  could  also  help  determine  the  research  program's  next  phase.  As  sug¬ 
gested  by  Boldoviei  and  Scott  (1984),  evaluators  should  thus  initially  deter¬ 
mine  the  CBI  system's  relative  efficiency  and  then  pinpoint  the  reasons  that 
the  system  was  more,  less,  or  equally  effective  than  was  the  comparative 
media.  The  evaluator's  role  is  to  assess  objectively  a  system's  instruction¬ 
al  value  and  not  to  help  with  the  design  process. 

Financial,  temporal,  and  personnel  limitations  may  prohibit  the  extensive 
evaluation  procedures  recommended  in  this  report.  Needless  CBI  studies  could 
be  eliminated  if  the  evaluation  team  could  build  upon  the  works  done  by  other 
teams  on  that  system.  Presently,  communications  problems  among  CBI  profes¬ 
sionals  present  an  obstacle  to  this  building  process.  Many  previous  evalua¬ 
tions  of  CBI  systems  are  not  easily  available.  Bangert-Drowns  et  al's  (1985) 
meta-analysis  of  the  CBI  literature  included  only  six  published  studies  out 
of  more  than  forty  studies  examined.  This  author  also  found  that  many  CBI 
reports  could  only  be  found  through  corporation  and  military  archives.  And 
some  studies  which  showed  unfavorable  results  regarding  a  CBI  system's  effec¬ 
tiveness  were  not  available  for  distribution. 

A  CBI  handbook,  such  as  Buros'  Mental  Measurement  Yearbooks,  is  needed  to 
provide  a  common  information  source  for  CBI  professionals  about  different  CBI 
systems  and  corresponding  evaluations.  This  handbook  should  include  a  de¬ 
tailed  description  of  the  studied  sample,  research  instruments,  research 
procedures,  and  findings.  Such  information  would  help  prospective  CBI 
implementers  to  determine  the  best  system  for  their  instructional  needs,  or 
to  discover  that  previous  CBI  evaluations  might  not  be  transferable  to  their 
situations.  The  military  has  tried  to  establish  such  a  CBI  library  with  the 


TRIADS  program  (see  Dallman  et  al.,  1983  for  a  description  of  this  program); 
however,  TRIADS  consists  of  a  library  of  CBI  programs  without  corresponding 
evaluation  data. 

Qiestions  do  remain  about  the  optimal  situations  for  implementing  a  CBI 
system.  As  indicated  throughout  this  report,  CBI  may  be  best  used  as  an 
instructional  tool  to  supplement  the  established  instructional  program  rather 
than  as  the  primary  instructional  medium.  Yet,  using  CBI  to  supplement 
rather  than  replace  other  instructional  modes  raises  further  questions  about 
the  costs  associated  with  this  medium.  If  CBI  can  help  problem  learners  than 
it  would  be  worth  the  additional  expenditures. 

Another  situational  aspect  involves  determining  the  appropriate  CBI  sys¬ 
tem  for  the  instructional  situation.  The  Army,  for  example,  must  decide  on 
such  options  as  using  a  hand-held  tutor  or  microcomputer  system  to  teach 
maintenance  students.  With  all  these  research  decisions,  evaluators  would  do 
better  to  shift  focus  from  questions  of  the  inherent  superiority  of  CBI  to 
the  identification  of  conditions  under  which  CBI  and  alternative  media  pro¬ 
duce  and  do  not  produce  the  desired  results.  Various  media  have  various 
strengths  which  must  be  first  enumerated  and  then  matched  with  intended  in¬ 
structional  settings,  objectives,  and  resources. 

Widespread  implementation  of  CBI  for  all  educational  situations  is  not 
recommended.  Such  indiscriminate  use  of  CBI  could  lead  to  the  abandonment  of 
this  instructional  technology,  because  the  research  evidence  has  indicated 
that  at  present  the  promises  may  outweigh  this  medium's  potential.  CBI  may 
be  useful  as  an  instructional  tool  which  can  help  make  established  instruc¬ 
tional  program  more  effective.  And  CBI  researchers  must  help  determine  those 
situations  for  which  instructors  can  best  use  this  medium. 


Conclusions 

In  closing,  CBI's  effectiveness  over  other  alternative  media  for  military 
training  has  yet  to  be  established .  CBI  can  provide  the  military  with 
standardization  of  instruction  and  provide  reserve  components  with  needed 
sustainment  training.  Also  future  generations  of  CBI,  especially  ICBI's, 
with  potential  advances  in  hardware,  software,  and  courseware  design  could 
make  this  medium  more  effective.  The  military  research  community  must  be 
more  careful  in  conducting  future  CBI  evaluations  and  must  be  careful  not  to 
overstate  the  promises  of  this  medium.  Researchers  of  military  training 
programs  should  also  focus  upon  determining  the  learning  tasks  most  suitable 
for  CBI,  and  should  concentrate  on  helping  the  military  education  community 
to  most  effectively  integrate  computers  in  already  established  instructional 
programs. 


Pinal  Implications 


1 .  Consistent  evidence  does  not  exist  to  support  the  instructional  superior¬ 
ity  of  CBI  over  other  media. 

2.  Future  generations  of  CBI,  such  as  ICBI,  promise  to  help  students'  cogni¬ 
tive  processes.  This  medium  also  promises  to  help  slow  learners.  Unfortu¬ 
nately,  problematic  research  practices  prohibit  any  conclusions  about  CBI  and 
slow  learners. 

3.  Problematic  research  procedures  are  found  throughout  this  literature. 

4.  The  methodological  recommendations  discussed  in  page  31  must  be  incorpo¬ 
rated  in  future  CBI  evaluation. 

5.  The  evaluation  team  ideally  would  consist  of  professionals  from  various 
disciplines. 

6.  Programmatic  research  on  CBI  systems  should  be  conducted  which  includes 
both  cross-sectional  and  longitudinal  studies. 

7.  Formative  evaluation  should  be  conducted  on  the  different  media  being 
compared. 

8.  Formative  evaluation  should  be  conducted  upon  completion  of  the  initial 
experimental  study. 

9.  A  CBI  handbook  should  be  developed  which  contains  information  about  pre¬ 
vious  CBI  evaluations. 

10.  CBI  researchers  should  shift  focus  from  examining  the  inherent  superior¬ 
ity  of  this  medium  to  identifying  conditions  for  using  computers  in  the  in¬ 
structional  process. 


11.  Widespread  implementation  of  CBI  for  all  educational  situations  is  not 
recommended . 
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